Funny behavior of readpartial in JRuby when reading a slow stream

[eli-darkly]

I'm groping in the dark here, I really have no idea at this point if the problem is JRuby or if I'm misusing httprb somehow, but here's the story.

Short version: in JRuby— and only in JRuby— when reading a chunked streaming response with httprb, readpartial sometimes fails if the chunks are 1 character long.

The context is that I'm testing a parser that operates on a chunked streaming response; I'm calling readpartial in a loop, and feeding each of the returned chunks into the parser's buffer. And in one of my tests, to make sure the parser can correctly handle the data no matter how incrementally it's delivered, I've set up the stream endpoint to write a single character at a time— flushing its output buffer and taking a slight delay in between each, to make it likely that the client will really be reading a character at a time rather than having it be buffered by the OS. So, what I should see on the client side is that readpartial returns a series of 1-character strings.

In a regular Ruby Linux environment, this always works fine.

In JRuby (both 9.2 and 9.3), sometimes it works. But other times, I get several correct 1-character chunks, but then readpartial starts returning a whole lot of zero-length (but not nil) chunks. And it never gives me the rest of the data— it's like the connection is permanently in a bad state once this happens.

In other tests where the same data is divided into bigger chunks, everything is fine. The problem only happens with 1-character chunks, and only in JRuby. Also, the data in this case is only ASCII characters and newlines, so multi-byte characters are not an issue.

Of course this is a somewhat contrived test scenario, and it's not my only way of testing the parser— I can easily test it with in-memory data without doing an HTTP request. But the HTTP logic will be used in real life, and I'm slightly concerned that there could be some condition (say, if the network gets very slow for a while) where something like this would happen in JRuby. So I'd welcome any thoughts.

(And, I will try to put together a minimal test case to see if others can reproduce this, but that will take a little while because I'll need to reproduce the server side as well. So in the meantime, I'm just hoping that maybe there's some known issue that will ring a bell with someone.)