netty-buffer throws IndexOutofBound or Null Pointer exception on bulk upload #7975
Comments
|
@sogillani were you able to work around the issue? |
|
I have the same issue openjdk 11.0.17 2022-10-18 |
|
@Dunemaster No. We are still facing this issue. We just has been able to avoid by limiting file size and number or files. Upload files in batches. |
|
@graemerocher are there any plans to fix it? We a considering Micronaut multipart mechanism and this issue currently blocks our usage Thank you! |
|
@yawkat is there any README to run application with 4.0.0? micronaut-parent is empty for 4.0.x branch and couldn't run application with core-bom. |
|
@sogillani , FYI, I replace Publusher with Publisher and it works fine |
|
@Dunemaster we are already using Publisher and it is not working for us. what is Publusher? I am not aware of it. Do you have any sample project on public repo? |
|
Look like github editor is broken
|
|
Ahaa! I had tried with Publisher<Publisher<byte[]>> data. It was working but got out of memory issue. |
Expected Behavior
Exposed a POST method in a Controller which accepts Publisher to upload around 1000 files of around 500 MB.
@post(consumes = MediaType.MULTIPART_FORM_DATA, produces = MediaType.APPLICATION_JSON)
In actual project we are uploading these files to Azure Blob store
Actual Behaviour
It throws following exception after uploading 10-20 files
java.lang.NullPointerException: Cannot read field "endOffset" because "this.components[...]" is null
at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:299)
Original Stack Trace:
at io.netty.buffer.CompositeByteBuf.addComponent0(CompositeByteBuf.java:299)
at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:265)
at io.netty.buffer.CompositeByteBuf.addComponent(CompositeByteBuf.java:222)
at io.netty.handler.codec.http.multipart.AbstractMemoryHttpData.addContent(AbstractMemoryHttpData.java:131)
at io.netty.handler.codec.http.multipart.MixedFileUpload.addContent(MixedFileUpload.java:109)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.loadDataMultipartOptimized(HttpPostMultipartRequestDecoder.java:1215)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.getFileUpload(HttpPostMultipartRequestDecoder.java:926)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.decodeMultipart(HttpPostMultipartRequestDecoder.java:572)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.parseBodyMultipart(HttpPostMultipartRequestDecoder.java:463)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.parseBody(HttpPostMultipartRequestDecoder.java:432)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.offer(HttpPostMultipartRequestDecoder.java:347)
at io.netty.handler.codec.http.multipart.HttpPostMultipartRequestDecoder.offer(HttpPostMultipartRequestDecoder.java:54)
at io.micronaut.http.server.netty.FormDataHttpContentProcessor.onData(FormDataHttpContentProcessor.java:148)
at io.micronaut.http.server.netty.AbstractHttpContentProcessor.doOnNext(AbstractHttpContentProcessor.java:79)
at io.micronaut.http.server.netty.AbstractHttpContentProcessor.doOnNext(AbstractHttpContentProcessor.java:36)
at io.micronaut.core.async.subscriber.CompletionAwareSubscriber.onNext(CompletionAwareSubscriber.java:56)
at io.micronaut.http.netty.reactive.HandlerPublisher.publishMessage(HandlerPublisher.java:393)
at io.micronaut.http.netty.reactive.HandlerPublisher.flushBuffer(HandlerPublisher.java:470)
at io.micronaut.http.netty.reactive.HandlerPublisher.publishMessageLater(HandlerPublisher.java:360)
at io.micronaut.http.netty.reactive.HandlerPublisher.channelRead(HandlerPublisher.java:323)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.micronaut.http.netty.stream.HttpStreamsHandler.handleReadHttpContent(HttpStreamsHandler.java:316)
at io.micronaut.http.netty.stream.HttpStreamsHandler.channelRead(HttpStreamsHandler.java:282)
at io.micronaut.http.netty.stream.HttpStreamsServerHandler.channelRead(HttpStreamsServerHandler.java:134)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
at io.netty.handler.codec.http.HttpServerKeepAliveHandler.channelRead(HttpServerKeepAliveHandler.java:64)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.flow.FlowControlHandler.dequeue(FlowControlHandler.java:200)
at io.netty.handler.flow.FlowControlHandler.read(FlowControlHandler.java:139)
at io.netty.channel.AbstractChannelHandlerContext.invokeRead(AbstractChannelHandlerContext.java:686)
at io.netty.channel.AbstractChannelHandlerContext.read(AbstractChannelHandlerContext.java:671)
at io.micronaut.http.netty.reactive.HandlerPublisher.requestDemand(HandlerPublisher.java:165)
at io.micronaut.http.netty.stream.HttpStreamsHandler$2.requestDemand(HttpStreamsHandler.java:273)
at io.micronaut.http.netty.reactive.HandlerPublisher$ChannelSubscription.receivedDemand(HandlerPublisher.java:556)
at io.micronaut.http.netty.reactive.HandlerPublisher$ChannelSubscription.lambda$request$0(HandlerPublisher.java:494)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
Steps To Reproduce
https://github.com/sogillani/upload
Run "mvn clean install" from the terminal. It would fail following unit test
Getting IndexOutOfBound in sample project
java.lang.IndexOutOfBoundsException: Index -1 out of bounds for length 1
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
at java.base/java.util.Objects.checkIndex(Objects.java:359)
at java.base/java.util.ArrayList.get(ArrayList.java:427)
at io.micronaut.http.server.netty.HttpDataReference.removeComponent(HttpDataReference.java:123)
at io.micronaut.http.server.netty.HttpDataReference$Component.lambda$getByteBuf$1(HttpDataReference.java:241)
at io.micronaut.http.server.netty.HttpDataReference$Component$1.release(HttpDataReference.java:199)
at io.micronaut.http.server.netty.multipart.NettyPartData.getBytes(NettyPartData.java:83)
at com.example.UploadController.lambda$upload$0(UploadController.java:37)
But actual project throws NullPointerException
Environment Information
OS: MAC
JDK: openjdk version "17.0.2" 2022-01-18
Actual project is running in kubernetes cluster
Example Application
https://github.com/sogillani/upload
Version
3.6.2
The text was updated successfully, but these errors were encountered: