Long connection eventually hangs

[cwc]

My use case is the consumption of server-sent events from a "firehose" API endpoint, which is constantly sending data. Ideally, my connection to this endpoint would never be closed.

The behavior I'm seeing is that, after an indeterminate amount of time, the connection hangs/freezes. No data is being received by my async client process, no errors are generated that I can see, and no %HTTPoison.AsyncEnd{} message has been received.

Whatever's happening seems to be related to hackney's recv_timeout option. With the default or a relatively low timeout like 30 seconds, the freeze happens fairly quickly - after a day or two. If I set the value of this option to :infinity, I see an uptime of two weeks or more - yet the freeze still eventually occurs.

Do I just need to set certain options to support my use case? Or is this a bug?

[sabondano]

I also need to use httpoison in the same manner and now I'm worried about it freezing. Any suggestions?

[edgurgel]

I'm not sure this is a hackney problem. It could be the way that the TCP connection is handled and its configuration options.

One approach would be to have some sort of "heartbeat" that if failed you simply restart the connection?

[cwc]

FWIW I haven't tried using hackney directly for this; haven't been able to find the time.

If it helps, here's the lib I wrote to consume server-sent events: https://github.com/cwc/eventsource_ex

[Fitblip]

Hey sorry for resurrecting a really old thread, but I ran into this recently with HTTPoison and streaming large files (to download large files).

I found that disabling the hackney pooling (hackney: [pool: false]) seems to resolve my issues:

HTTPoison.get!(
    url, 
    ["Authorization": "Bearer #{token}"], 
    [stream_to: self(), async: :once, hackney: [pool: false]]
)

I'm going to update the other thread with the same (#161 hopefully that's OK).