Skip to content

Commit

Permalink
fix (downloads): Workaround for Hackney issue where, after too many f…
Browse files Browse the repository at this point in the history
…ailed downloads, the pool is (permanently?) exhausted

Without this, after we hit a few HTTP errors (which of course NOAA serves us a lot, that's the whole point of this proxy), all (?) future downloads fail with this error:
{:error, %HTTPoison.Error{id: nil, reason: :checkout_timeout}}

Hackney issue: edgurgel/httpoison#414
More info: benoitc/hackney#643
  • Loading branch information
s3cur3 committed Oct 23, 2020
1 parent daaec26 commit 8df2276
Showing 1 changed file with 11 additions and 0 deletions.
11 changes: 11 additions & 0 deletions lib/weather_mirror/auto_updating_url_cache.ex
Expand Up @@ -80,6 +80,17 @@ defmodule WeatherMirror.AutoUpdatingUrlCache do
|> strip_unwanted_noaa_headers()
|> add_cached_header()

{:error, %HTTPoison.Error{reason: :checkout_timeout}} ->
Logger.warn("Failed to grab #{url} due to :checkout_timeout error; rebooting pool")
# Workaround for https://github.com/edgurgel/httpoison/issues/414
# Without this, after we hit a few HTTP errors (which of course NOAA serves
# us a lot, that's the whole point of this proxy), all (?) future downloads
# fail with this error:
# {:error, %HTTPoison.Error{id: nil, reason: :checkout_timeout}}
# More info: https://github.com/benoitc/hackney/issues/643
:hackney_pool.stop_pool(:default)
prev_response

e ->
Logger.warn("Failed to grab #{url}\n#{inspect(e)}")
prev_response
Expand Down

0 comments on commit 8df2276

Please sign in to comment.