Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terraform output should adhere to Terraform Cloud authorization #31507

Merged
merged 7 commits into from Jul 28, 2022

Conversation

brandonc
Copy link
Contributor

@brandonc brandonc commented Jul 25, 2022

Normally, terraform output refreshes and reads the entire state in the command package before pulling output values out of it. This doesn't give Terraform Cloud the opportunity to apply the "read state outputs" org permission and instead requires the elevated "read state versions" permission.

I decided to expand the state manager interface to provide a separate GetRootOutputValues function in order to give the cloud backend a more nuanced opportunity to fetch just the outputs. This required moving state Refresh/Read code that was previously in the command into the shared backend state as well as the filesystem state packages.

Notes

Since terraform is now consuming outputs from the API, the API supports a new "detailed-type" output attribute (depending on feature flags) which contains the terraform type encoded as the terraform-json tuple. The type and value are then used to recreate the output type with a full cty Value that should give identical results to the local state version.

Testing Steps

  1. Create a new workspace on TFC with local only mode (state storage only)
  2. Apply the example config below (in the new workspace) with different kinds of outputs on the new workspace you created.
  3. Use different forms of terraform output - you should see the same results as before.
Example Config
terraform {
  backend "remote" {
    hostname = "app.staging.terraform.io"
    token = "bcroft"
    organization = "bcroft"
workspaces {
  name = "test-json-outputs"
}

}
}

resource "null_resource" "test" {
triggers = {
hello = "wat11"
}
}

resource "random_id" "random" {
keepers = {
uuid = uuid()
}

byte_length = 8
}

output "an-object" {
value = {
nesting1: {
nesting2: {
nesting3: null_resource.test.id
}
}
}
}

output "random" {
value = random_id.random.hex
description = "a random hex output"
}

output "a-true-bool" {
value = true
description = "The true of the main server instance."
}

output "escapes" {
value = "line 1\nline 2\n\\\\\n"
description = "The true of the main server instance."
}

output "a-false-bool" {
value = false
description = "The false of the main server instance."
}

output "a-string" {
value = "example string"
description = "The private string of the main server instance."
}

output "an-int" {
value = 1001
description = "The private integer of the main server instance."
}

output "a-decimal" {
value = 1000.1
description = "The private decimal of the main server instance."
}

output "a-list" {
value = tolist(["example", 1001, 1000.1])
description = "The list"
}

output "a-map" {
value = tomap({ "example" : 1001, "bar" : 1000.1 })
description = "The map"
}

output "a-tuple" {
value = ["1", "example"]
description = "The tuple"
}

output "a-long-string" {
value = "The private integer of the main server instance is where you want to go when you have the most fun in every Terraform instance you can see in the world that you live in except for dogs because they don't run servers in the same place that humans do."
description = "A long string is good for you."
}

output "a-sensitive-value" {
value = "hopefully you cannot see me"
description = "The sensitive string of the main server instance."
sensitive = true
}

Example output

$ terraform output
a-decimal = 1000.1
a-false-bool = false
a-list = tolist([
  "example",
  "1001",
  "1000.1",
])
a-long-string = "The private integer of the main server instance is where you want to go when you have the most fun in every Terraform instance you can see in the world that you live in except for dogs because they don't run servers in the same place that humans do."
a-map = tomap({
  "bar" = 1000.1
  "example" = 1001
})
a-sensitive-value = <sensitive>
a-string = "example string"
a-true-bool = true
a-tuple = [
  "1",
  "example",
]
an-int = 1001
an-object = {
  "nesting1" = {
    "nesting2" = {
      "nesting3" = "6032582464784446680"
    }
  }
}
escapes = <<EOT
line 1
line 2
\\\\

EOT
random = "b55d20555dbfb5f1"

$ terraform output a-sensitive-value
"hopefully you cannot see me"

$ terraform output -json
...

When reading state outputs from earlier versions of terraform (that do not send the detailed type information) terraform will fall back to reading the entire state file. If that fails, a friendly error is shown:

$ TF_TOKEN_app_terraform_io="some-team-token-that-can-only-read-state-outputs" terraform output

│ Error: You are not authorized to read the full state version containing outputs.
│ State versions created by terraform v1.3.0 and newer do not require this level
│ of authorization and therefore this error can be fixed by upgrading the remote
│ state version.
│
│

TODO: validate that users that cannot read the full state see a 403 forbidden error instead of a 404 not found error

Normally, `terraform output` refreshes and reads the entire state in the command package before pulling output values out of it. This doesn't give Terraform Cloud the opportunity to apply the read state outputs org permission and instead applies the read state versions permission.

I decided to expand the state manager interface to provide a separate GetRootOutputValues function in order to give the cloud backend a more nuanced opportunity to fetch just the outputs. This required moving state Refresh/Read code that was previously in the command into the shared backend state as well as the filesystem state packages.
@@ -70,6 +73,22 @@ func (s *State) WriteState(state *states.State) error {
return s.delegate.WriteState(state)
}

func (s *State) fallbackReadOutputsFromFullState() (map[string]*states.OutputValue, error) {
if err := s.RefreshState(); err != nil {
if strings.HasSuffix(err.Error(), "failed to retrieve state: forbidden") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is probably resource not found in production 🤔

Copy link
Member

@jbardin jbardin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! I don't necessarily like the additional interface here, but the method would probably look out of place given the existing patterns, and we can work on refactoring all these those at a later time.

@brandonc
Copy link
Contributor Author

@jbardin Thanks, James! When you say existing patterns do you mean the Persistent interface? Is there anything I can do to improve this now, such as move my new method to the state Reader interface?

@jbardin
Copy link
Member

jbardin commented Jul 28, 2022

I wouldn't worry about it. We need to refactor the state/backend abstractions in general, so let's just leave it for now. The individual composed interfaces were never really useful, and make reading which interface does what or deciding which to use more difficult, but I think breaking from the existing pattern for a new single method would only be more confusing.

@brandonc brandonc merged commit c62e20c into main Jul 28, 2022
@brandonc brandonc deleted the brandonc/output_cloud_reads branch July 28, 2022 17:54
@github-actions
Copy link

Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch.

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants