Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GET not following 307 redirects #264

Open
1 task done
louhnmn opened this issue May 8, 2023 · 1 comment
Open
1 task done

GET not following 307 redirects #264

louhnmn opened this issue May 8, 2023 · 1 comment
Labels
bug

Comments

@louhnmn
Copy link

louhnmn commented May 8, 2023

Terraform CLI and Provider Versions

Terraform v1.4.6
on darwin_amd64

  • provider registry.terraform.io/hashicorp/http v3.3.0

Terraform Configuration

terraform {
  required_version = ">= 1.4.0"
}

variable "gsheet" {
  type = object({
    base_url = string
    pub_id   = string
    gid      = string
  })
  default = {
    base_url = "https://docs.google.com/spreadsheets/d/e/"
    pub_id   = "2PACX-1vRzZjUzim7pcO9Q3xBOCf9_GmMEANBLVeGpPbUOX-GzfQLU6bIgEkOQsXOuuM-M9ephWi3TSSO9vpWH"
    gid      = "0"
  }
}

data "http" "get_sheet" {
  url    = format("%s%s/pub?gid=%s&output=csv", var.gsheet.base_url, var.gsheet.pub_id, var.gsheet.gid)
  method = "GET"

  request_headers = {
    Accept = "text/csv"
  }
}

output "sheet_response_body" {
  value = data.http.get_sheet.response_body
}

Expected Behavior

csv data should be output.

colA colB
123 456
789 12

Actual Behavior

a 307 redirect response html is output.

<HTML>
        <HEAD>
        <TITLE>Temporary Redirect</TITLE>
        </HEAD>
        <BODY BGCOLOR="#FFFFFF" TEXT="#000000">
        <H1>Temporary Redirect</H1>
        The document has moved <A HREF="https://doc-0s-88-sheets.googleusercontent.com/pub/n4743ueaoiib6is67cgugk097k/dd0k4os4p03bpv4iksdjr6h7gg/1683540870000/105210521312657442095/*/e@2PACX-1vRzZjUzim7pcO9Q3xBOCf9_GmMEANBLVeGpPbUOX-GzfQLU6bIgEkOQsXOuuM-M9ephWi3TSSO9vpWH?gid=0&amp;output=csv">here</A>.
        </BODY>
</HTML>

Steps to Reproduce

  1. terraform plan

How much impact is this issue causing?

High

Logs

No response

Additional Information

public link of the google sheet:
sheet

With the Location parameter set (-L) the 307 redirect will success.

A curl -Li will output:

HTTP/2 307 
content-type: text/html; charset=UTF-8
cache-control: no-cache, no-store, max-age=0, must-revalidate
pragma: no-cache
expires: Mon, 01 Jan 1990 00:00:00 GMT
date: Thu, 04 May 2023 10:33:15 GMT
location: https://doc-0s-88-sheets.googleusercontent.com/pub/n4743ueaoiib6is67cgugk097k/8ae3sg2p0gv6016ai6lm64n7p4/1683196395000/105210521312657442095/*/e@2PACX-1vRzZjUzim7pcO9Q3xBOCf9_GmMEANBLVeGpPbUOX-GzfQLU6bIgEkOQsXOuuM-M9ephWi3TSSO9vpWH?gid=0&single=true&output=csv
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
server: GSE
set-cookie: NID=511=llanR2tu9k6ac9<...redacted...>_QuFpih9Z5Om4; expires=Fri, 03-Nov-2023 10:33:14 GMT; path=/; domain=.google.com; HttpOnly
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
accept-ranges: none
vary: Accept-Encoding

HTTP/2 200 
content-type: text/csv
x-frame-options: ALLOW-FROM https://docs.google.com
x-robots-tag: noindex, nofollow, nosnippet
expires: Thu, 04 May 2023 10:33:18 GMT
date: Thu, 04 May 2023 10:33:18 GMT
cache-control: private, max-age=300
content-disposition: attachment; filename="test_table-data1.csv"; filename*=UTF-8''test_table%20-%20data1.csv
access-control-allow-origin: *
access-control-expose-headers: Cache-Control,Content-Disposition,Content-Encoding,Content-Length,Content-Type,Date,Expires,Server,Transfer-Encoding
content-security-policy: frame-ancestors 'self' https://docs.google.com
content-security-policy: base-uri 'self';object-src 'self';report-uri https://doc-0s-88-sheets.googleusercontent.com/spreadsheets/cspreport;script-src 'nonce-e1_RvbtZa_0Z_GdQwIpG9g' 'unsafe-inline' 'strict-dynamic' https: http: 'unsafe-eval';worker-src 'self' blob:
x-content-type-options: nosniff
x-xss-protection: 1; mode=block
server: GSE
alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
accept-ranges: none
vary: Accept-Encoding

colA,colB
123,456
789,12

Code of Conduct

  • I agree to follow this project's Code of Conduct
@louhnmn louhnmn added the bug label May 8, 2023
@louhnmn
Copy link
Author

louhnmn commented Jul 24, 2023

I actually found a workaround last week just by clicking around on the response link.
So the 307 redirect response contains the redirect url in the <A HREF="...url_is_here...">...</A> tag.
It will be valid for a few seconds (something like 5 seconds).
So now we can just extract this url and then put a second http data-source around it and you are good to go.

so this would be the final code assuming the variables are already set up:

# first data source that will lead to the redirect response
data "http" "get_sheet" {
  url    = format("%s%s/pub?gid=%s&output=csv", var.gsheet.base_url, var.gsheet.pub_id, var.gsheet.gid)
  method = "GET"

  request_headers = {
    Accept = "text/csv"
  }
}

# a locals variable that extracts the redirect url from the first response
locals {
  sheet_redirect_url = replace(regex("<A HREF=\"(.+)\">here</A>", data.http.get_sheet_normal_unique.response_body)[0], "&amp;", "&")
}

# output just to see if the url looks good
output "sheet_extracted_redirect_url" {
  value = local.sheet_redirect_url
}

# second data source to get the actual csv (or whatever data)
data "http" "get_sheet_redirected" {
  url    = local.sheet_redirect_url
  method = "GET"

  request_headers = {
    Accept = "text/csv"
  }
}

# output to see the data
output "sheet_response_redirected" {
  value = data.http.get_sheet_redirected.response_body
}

@louhnmn louhnmn mentioned this issue Jul 24, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@louhnmn