NullPointerException in JSON codec parsing stream encoded with Jackson Smile

[eftche]

Affects: 5.1.11

Using Spring 5.1.11, Spring Boot 2.1.10, Jackson 2.10.1, but it was there in previous versions as well.

There seems to be an issue in Spring Web with #20699, streaming Smile encoded data.

I'm trying to receive a stream of objects created by Spring WebFlux app as Flux<T> using Smile encoding. JSON stream works OK.

With Smile most of the time I get NullPointerException:

at com.fasterxml.jackson.databind.util.TokenBuffer.copyCurrentEvent(TokenBuffer.java:1032)
at org.springframework.http.codec.json.Jackson2Tokenizer.processTokenArray(Jackson2Tokenizer.java:170)
at org.springframework.http.codec.json.Jackson2Tokenizer.parseTokenBufferFlux(Jackson2Tokenizer.java:135)
at org.springframework.http.codec.json.Jackson2Tokenizer.tokenize(Jackson2Tokenizer.java:94)

It seems that current token of parser passed to copyCurrentEvent is null.

I'm trying now to understand how exactly does it happen but decided to raise the issue anyway.

Unfortunately it's not the easiest one to debug as it goes away if you slow it down, which probably suggests that the problem manifests itself when there's no input available to process as next array element is not received yet.

[poutsma]

After some digging through the stack trace, it seems like the null pointer occurs at this line: https://github.com/FasterXML/jackson-databind/blob/master/src/main/java/com/fasterxml/jackson/databind/util/TokenBuffer.java#L1037.

So it's not the token that is null, nor the JsonParser passed to copyCurrentEvent, but rather the currentToken of the parser that returns null.

Obviously we can check whether the parser has a current token with hasCurrentToken before copying the event, but I'd like to be able to check if that fixes this issue completely. So if you'd like us to spend some time investigating, please take the time to provide a complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem.

[eftche]

test-spring-smile.zip
Attached example reproduces the issue for me on my not that fast laptop.
If it doesn't fail for you on faster machine you might need to tighten up the timing between produced events to increase the pressure.

[poutsma]

Thanks for the example, I am able to reproduce the issue. I will investigate further.