Getting redirected only when using Excon, not with other HTTP clients

[roback]

I get redirected when using excon to get the following url: http://defensacentral.com/rss/rss.php?sec=0. When using typhoeus or net/http (or a browser) the server responds with status 200.

I tried to figure out what excon is doing differently compared to the other http clients, but I cannot find the cause behind this.

The url above is the only one where I have noticed this behaviour, but maby there are more.

Example:

pry> require "excon"
=> true
pry> require "typhoeus"
=> true
pry> require "net/http"
=> true
pry> url = "http://defensacentral.com/rss/rss.php?sec=0"
=> "http://defensacentral.com/rss/rss.php?sec=0"
pry> Typhoeus.get(url).response_code
=> 200
pry> Net::HTTP.get_response(URI(url)).code
=> "200"
pry> Excon.get(url).status
=> 301

# Response headers (typhoeus vs excon)

pry> Typhoeus.get(url).headers
=> {"Date"=>"Mon, 21 Sep 2015 06:50:47 GMT",
 "Content-Type"=>"text/xml;charset=ISO-8859-1",
 "Transfer-Encoding"=>"chunked",
 "Connection"=>"keep-alive",
 "Set-Cookie"=>"__cfduid=d6798dbde9ff0bb5be616ceb9cca711611442818247; expires=Tue, 20-Sep-16 06:50:47 GMT; path=/; domain=.defensacentral.com; HttpOnly",
 "Vary"=>"Host",
 "Cache-Control"=>"max-age=0",
 "Expires"=>"Mon, 21 Sep 2015 06:50:48 GMT",
 "Server"=>"cloudflare-nginx",
 "CF-RAY"=>"2293d67f5cfa16b2-ARN"}
pry> Excon.get(url).headers
=> {"Date"=>"Mon, 21 Sep 2015 06:50:18 GMT",
 "Content-Type"=>"text/html; charset=iso-8859-1",
 "Connection"=>"keep-alive",
 "Set-Cookie"=>"__cfduid=dacd7401cc3e2bdc68ec2e9e9684d134f1442818218; expires=Tue, 20-Sep-16 06:50:18 GMT; path=/; domain=.defensacentral.com; HttpOnly",
 "Location"=>"http:///rss/rss.php?sec=0",
 "Cache-Control"=>"max-age=0",
 "Expires"=>"Mon, 21 Sep 2015 06:50:18 GMT",
 "Server"=>"cloudflare-nginx",
 "CF-RAY"=>"2293d5c7569a168e-ARN"}

[dentarg]

Maybe it has something to do with compression

I tried using the Decompress middleware:

require "excon"
middlewares = Excon.defaults[:middlewares] + [Excon::Middleware::RedirectFollower] + [Excon::Middleware::Decompress]
connection = Excon.new(url, middlewares: middlewares)
url = "http://defensacentral.com/rss/rss.php?sec=0"
connection.get.status

and I got

SystemStackError: stack level too deep
from /Users/dentarg/.gem/ruby/2.2.2/gems/excon-0.45.3/lib/excon/socket.rb:137:in `==='

[dentarg]

Maybe it has something to do with compression

I spotted that Typhoeus had "Vary"=>"Host", above and Excon not.

[geemus]

It probably has to do with excon including default ports. You can try with :omit_default_port => true set and you may have better luck. We'll probably just change this at some point, but I've been reticent to do so in case the opposite case might also break things.

[roback]

You can try with :omit_default_port => true set and you may have better luck.

Yes, that works 😄 Thanks.

I'll close this as there are other issues discussing the :omit_default_port option (such as #357).

[geemus]

Sounds good, sorry about the inconvenience, but glad it was an easy fix.