Host header is not sent with request_headers

[taeris]

Terraform CLI and Provider Versions

Terraform v1.4.6
on linux_amd64

• provider registry.terraform.io/hashicorp/http v3.4.0

Terraform Configuration

data "http" "example" {
  url = "https://xxx.xxx.xxx.xxx"

  request_headers = {
    Host = "www.example.com"
  }
}

Expected Behavior

I expect the Host header www.example.com to be sent as part of the request headers to the server and a 200 response/body sent back, when overriding the Host header using the request_headers block.

Actual Behavior

The Host header is not sent as part of the request, but appears to be stripped from the request.
For https requests the following is returned with insecure = false:
Error making request: GET https://xxx.xxx.xxx.xxx giving up after 1 attempt(s): Get "https://xxx.xxx.xxx.xxx": x509: cannot validate certificate for xxx.xxx.xxx.xxx because it doesn't contain any IP SANs

With insecure = true a 404 response is returned.

Steps to Reproduce

1. terraform apply

How much impact is this issue causing?

Medium

Logs

No response

Additional Information

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[bendbennett]

Hi @taeris 👋

Local testing indicates that the Host is set directly on the http.Request object (i.e., r.Host in the following example).

func TestDataSource_withHostHeader_200(t *testing.T) {
	testServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if strings.Contains(r.Host, "127.0.0.1") {
			w.WriteHeader(http.StatusOK)
		} else {
			w.WriteHeader(http.StatusNotFound)
		}
	}))
	defer testServer.Close()

	resource.ParallelTest(t, resource.TestCase{
		ProtoV5ProviderFactories: protoV5ProviderFactories(),
		Steps: []resource.TestStep{
			{
				Config: fmt.Sprintf(`
							data "http" "http_test" {
								url = "%s"
							}`, testServer.URL),
				Check: resource.ComposeTestCheckFunc(
					resource.TestCheckResourceAttr("data.http.http_test", "status_code", strconv.Itoa(http.StatusOK)),
				),
			},
		},
	})
}

The docs state the following:

	// Header contains the request header fields either received
	// by the server or to be sent by the client.
	//
	// If a server received a request with header lines,
	//
	//	Host: example.com
	//	accept-encoding: gzip, deflate
	//	Accept-Language: en-us
	//	fOO: Bar
	//	foo: two
	//
	// then
	//
	//	Header = map[string][]string{
	//		"Accept-Encoding": {"gzip, deflate"},
	//		"Accept-Language": {"en-us"},
	//		"Foo": {"Bar", "two"},
	//	}
	//
	// For incoming requests, the Host header is promoted to the
	// Request.Host field and removed from the Header map.

Can you expand a little further on your requirement for setting the Host header if the above is not helpful in your use-case?

[azafairon]

Hello @bendbennett

I also facing the same issue. If a make a request to an NGINX server using the following :

data "http" "test" {
  url      = "https://x.x.x.x/path"
  insecure = true
  request_headers = {
    Host = "hostname"
  }
}

NGINX will respond with 404 because can not figure out the location (host). However doing the same request using curl :
curl -skv -H "Host: hostname" https://x.x.x.x/path

everything is fine.

I think somehow the promotion of Host header to request.Host is lost.

Thanks

[taeris]

Hi @bendbennett and @azafairon ,

Thanks for taking the time to reply to my bug report.
Re-reading my original ticket I think I omitted some key information in my effort to be as concise as possible, so apologies for that.

The host header is available when using a normal http resource, but what is not being applied is a host header override when set in the request_headers block.

So if I setup the following data resource:

data "http" "test" {
  url      = "https://example.com"
}

the host header will be correctly sent.

If on the other hand you setup the following resource:

data "http" "test2" {
  url      = "https://www.example.com"
  request_headers = {
    Host = "www.differentexample.com"
  }
}

the Host header set inside the request_headers block won't override the Host header that is being set up by the url part of the resource.

What I'm trying to solve is to use an http resource to send a request to a server that has a load balancer deployed that expects a domain in the host header, but that does not actually have a DNS record created yet as it's still being bootstrapped. Updating the hosts file is not something that we can do, so we were hoping to be able to override the host header as can be seen in the curl example above.

I hope the additional information make it a bit clearer

Thanks again.

[bendbennett]

Hi @taeris 👋

Thank you for the additional context.

I believe that to achieve what you describe would require an alteration to the provider code that would allow setting request.Host on the basis of Host being set in the request_headers in the supplied configuration.

This would represent a change to the current behaviour of the provider so would likely require some sort of "opt-in" mechanism, such as introducing a configurable field indicating that the request.Host should be set with the supplied Host value in the request_headers, or something along those lines.

Perhaps we can leave this issue open and assess the level of community interest on the basis of up-votes to determine prioritisation.

[taeris]

Hi @bendbennett ,

Thanks again for replying.
I've updated the bug report with a bit more details in the Expected behavior block after our discussion above.

In regards to adding an opt-in flag, wouldn't the actual setting of the Host header in request_block provide that? As I see it, that block would be used when you're trying to add/replace headers that are missing or generated for you by the code. It is an implementation detail so I guess something to be considered when the issue is worked on?

Agreed upon leaving the issue open to assess interest. Hopefully there will be some :)

Thanks again.

[mattias-fjellstrom]

I'm in a situation where I am trying to use the HTTP data source in combination with the Terraform test framework. In a given test I want to verify the behaviour of an Azure Traffic Manager DNS load balancer. I must send the Host header to be able to verify that the different endpoints are set up correctly. Spent a lot of time debugging why it was not working, and then I found this issue :)