Port 80 is included in the 'Host' request field (instead of defaulted) when it isn't with other popular HTTP clients #515

celer · 2013-04-15T21:54:42Z

Hi,

So we encountered an interesting problem today, less so a bug with request but more of a incongruent behavior from request when compared against other browsers and client's we've tested.

When working with cloudfoundry.com we found that their proxy did not consider a request to port 80 the same if the port was explicitly specified in the 'Host' header field.

So a request to http://foo.cloudfoundry.com:80/ and http://foo.cloudfoundry.com/ behaved differently. We narrowed it down to the difference in the Host field. Request included the port when explicitly specifying it, but not when it wasn't.

http://foo.cloudfoundry.com:80/ has a Host header of "Host: foo.cloudfoundry.com:80"
http://foo.cloudfoundry.com/ has a Host header of "Host: foo.cloudfoundry.com"

No big deal right? Well so everyone else strips port 80 because it is the default :(

Curl strips port 80 from the Host header
wget strips port 80 from the Host header
Safari strips port 80 from the Host header
Firefox strips port 80 from the Host header
Chrome strips port 80 from the Host header

You can test the behavior using "http://helloworld-ng.cloudify.com:80/" and "http://helloworld-ng.cloudify.com/" - notice that both URLs work the same in all of the above listed browsers

In curl we can see the failure case

pair@foo:~pair/$ curl http://helloworld-ng.cloudfoundry.com:80/ -H "Host: helloworld-ng.cloudfoundry.com:80"
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
pair@foo:~pair/$ curl http://helloworld-ng.cloudfoundry.com:80/ -H "Host: helloworld-ng.cloudfoundry.com"
Hello from Cloud Foundry

Now let's try it with request

request=require('request');

request.get("http://helloworld-ng.cloudfoundry.com:80/",function(err,req,body){
  console.log("this will fail with port 80",body);
});

request.get("http://helloworld-ng.cloudfoundry.com/",function(err,req,body){
  console.log("this will not fail without port 80",body);
});

(We've asked cloudfoundry to fix this issue, but to be clear this is incongruent with every other client and browser we've tested)

So then the question becomes - should request do the same? Sadly it seems like this shouldn't be an issue but it is an unexpected behavior and no one else does it - hence why I think request should strip port 80 from the Header field if it is specified.

On that note node-http-proxy suffers form the same problem and a nice bit of logic that it uses some places to address it:

  var host=host+(port-80===0?"":":"+port)

mikeal · 2013-04-15T23:01:17Z

hrm.....

do all those applications strip it even when you put in the full url with :80?

celer · 2013-04-16T14:52:13Z

Hi Mikeal,

Yes, every client I have strips port 80 even when specified, see below for packet captures (using ngrep).

Chrome request to http://helloworld-ng.cloudfoundry.com:80/

T 192.168.2.3:55280 -> 173.243.49.35:80 [AP]
  GET / HTTP/1.1..Host: helloworld-ng.cloudfoundry.com..Connection: keep-alive..Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8..User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleW
  ebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.65 Safari/537.31..Accept-Encoding: gzip,deflate,sdch..Accept-Language: en-US,en;q=0.8..Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3..Cookie: __qca=P0-2133402613-136
  6064815021....

Firefox request to http://helloworld-ng.cloudfoundry.com:80/

T 192.168.2.3:55274 -> 173.243.49.35:80 [AP]
  GET / HTTP/1.1..Host: helloworld-ng.cloudfoundry.com..User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:20.0) Gecko/20100101 Firefox/20.0..Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q
  =0.8..Accept-Language: en-US,en;q=0.5..Accept-Encoding: gzip, deflate..Connection: keep-alive....

Safari request to http://helloworld-ng.cloudfoundry.com:80/

T 192.168.2.3:55289 -> 173.243.49.35:80 [AP]
  GET / HTTP/1.1..Host: helloworld-ng.cloudfoundry.com..User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10..Accept: text/html,application
  /xhtml+xml,application/xml;q=0.9,*/*;q=0.8..Accept-Language: en-us..Accept-Encoding: gzip, deflate..Connection: keep-alive....

Curl request to http://helloworld-ng.cloudfoundry.com:80/

T 192.168.2.3:55297 -> 173.243.49.35:80 [AP]
  GET / HTTP/1.1..User-Agent: curl/7.30.0..Host: helloworld-ng.cloudfoundry.com..Accept: */*....

wget request to http://hellloworld-ng.cloudfoundry.com:80/

T 192.168.2.3:55303 -> 173.243.49.35:80 [AP]
  GET / HTTP/1.1..User-Agent: Wget/1.14 (darwin12.3.0)..Accept: */*..Host: helloworld-ng.cloudfoundry.com..Connection: Keep-Alive....

Here is the response we got from cloudfoundry regarding this issue:

Hi, you're are correct and obviously what should happen is the port on the header should be stripped. 
However I am not totally sure that if you were to request http://helloworld-ng.cloudfoundry.com:80 
from a browser, the port would be included in the host header. I have also tested this on a more 
recent version of Cloud Foundry and it's not an issue, so this should change when the new version 
of .com is released.

Regardless of the response from Cloudfoundry, this is still a bug / incongruence in request.

Thanks,

celer

celer · 2013-04-17T14:39:43Z

I also verified the behavior with HTTPS when using port 443. All the clients strip 443 from the host field when using https.

I used a simple nodejs server and the various development tools with the browsers to verify the behavior.

var fs = require('fs');
var http = require('https');
var privateKey  = fs.readFileSync('privatekey.pem').toString();
var certificate = fs.readFileSync('certificate.pem').toString();

var credentials = {key: privateKey, cert: certificate};
var express = require('express');
var server = express();

var app = express();

app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(app.router);

app.get("/",function(req,res){
  console.log(req.headers);
  console.log("Got /");
  res.send("hello");
});

var https = http.createServer(credentials, app);

https.listen(443);

mikeal · 2013-04-17T16:35:48Z

right, but the pull request you sent doesn't handle https.

joshenders · 2013-06-21T17:26:04Z

This doesn't appear to be a violation of the RFC:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23

Setting default port in Host header break some web servers like "Apache Coyote". Most clients don't set this default port too: * request/request#515 (comment) * https://github.com/ruby/ruby/blob/d46336e71731aa64d71d4573b2741b7de43ec340/lib/net/http/generic_request.rb#L18

When `http://EXAMPLE:80` is an allowed origin, requests are not allowed from `http://EXAMPLE`. Since port 80 is the default port for HTTP, browsers will strip it and thus rack-cors never receives a request from `http://EXAMPLE`. A similar problem is discussed here: request/request#515

…wn. (#2904) * Strip port suffix from Host header if the protocol is known. This partially revert ff6d6c6, and still works for IPv6 addresses as well. * Port is a string out of url.parse(). See https://nodejs.org/api/url.html#url_url_port * Port is a string out of url.parse(). See https://nodejs.org/api/url.html#url_url_port * Add tests for the new Host header changes.

celer mentioned this issue Apr 16, 2013

To be congruent with other popular http clients the port should be stripped from the 'Host' field when the port is 80. http-party/node-http-proxy#404

Closed

mikeal closed this as completed Aug 28, 2014

splattael mentioned this issue Jun 9, 2015

Omit port in Host header for default ports jaswope/rack-reverse-proxy#51

Closed

waterlink mentioned this issue Jul 3, 2015

Omit port in Host header for default ports waterlink/rack-reverse-proxy#3

Merged

bryevdv mentioned this issue Jan 3, 2016

accept implicit port 80 on whitelist bokeh/bokeh#3513

Merged

lrstanley mentioned this issue Apr 27, 2016

Exclude port 443 from host http header websocket-client/websocket-client#248

Merged

karrots mentioned this issue Jun 15, 2016

HTTP Host header standard ports nodemcu/nodemcu-firmware#1361

Closed

paambaati mentioned this issue Mar 27, 2018

'Host' header contains :80 or :443 node-modules/urllib#280

Closed

This was referenced Apr 3, 2018

header field "host" includes port 80 #2894

Closed

#515, #2894 Strip port suffix from Host header if the protocol is known. #2904

Merged

nbr mentioned this issue Apr 4, 2018

add note on origins w/ default port to README.md cyu/rack-cors#163

Open

fviard mentioned this issue Mar 19, 2020

SignatureDoesNotMatch error if port 80 is used and port is indicated in Host header minio/minio#9169

Closed

This was referenced Mar 16, 2021

Bump request from 2.34.0 to 2.88.2 helenkulka/spotify-taste-tester#38

Open

Bump request from 2.49.0 to 2.88.2 baconjs/baconjs.github.io#51

Closed

michel-kraemer mentioned this issue Aug 17, 2022

Corrupted downloaded file michel-kraemer/gradle-download-task#241

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Port 80 is included in the 'Host' request field (instead of defaulted) when it isn't with other popular HTTP clients #515

Port 80 is included in the 'Host' request field (instead of defaulted) when it isn't with other popular HTTP clients #515

celer commented Apr 15, 2013

mikeal commented Apr 15, 2013

celer commented Apr 16, 2013

celer commented Apr 17, 2013

mikeal commented Apr 17, 2013

joshenders commented Jun 21, 2013

Port 80 is included in the 'Host' request field (instead of defaulted) when it isn't with other popular HTTP clients #515

Port 80 is included in the 'Host' request field (instead of defaulted) when it isn't with other popular HTTP clients #515

Comments

celer commented Apr 15, 2013

mikeal commented Apr 15, 2013

celer commented Apr 16, 2013

celer commented Apr 17, 2013

mikeal commented Apr 17, 2013

joshenders commented Jun 21, 2013