Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow TF_LOG/TF_LOG_CORE/TF_LOG_PROVIDER to be set correctly in Acceptance Tests #992

Closed
megan07 opened this issue Jun 30, 2022 · 2 comments · Fixed by #993
Closed

Allow TF_LOG/TF_LOG_CORE/TF_LOG_PROVIDER to be set correctly in Acceptance Tests #992

megan07 opened this issue Jun 30, 2022 · 2 comments · Fixed by #993
Assignees
Labels
enhancement New feature or request subsystem/tests Issues and feature requests related to the testing framework.
Milestone

Comments

@megan07
Copy link
Contributor

megan07 commented Jun 30, 2022

SDK version

2.16.0

Use-cases

Recently we have been seeing these core logs in our tests. They show up locally as well. I can turn them off if TF_LOG and TF_LOG_CORE are not set, but then also our provider logs are gone. I tried setting TF_LOG_PROVIDER=DEBUG but that didn't work and later noticed that doesn't work for acceptance tests. (See slack context).

Attempted Solutions

Proposal

References

@megan07 megan07 added the enhancement New feature or request label Jun 30, 2022
@bflad bflad added this to the v2.18.0 milestone Jun 30, 2022
@bflad bflad self-assigned this Jun 30, 2022
bflad added a commit that referenced this issue Jun 30, 2022
…erraform during acceptance testing on versions 0.15 and later

Reference: hashicorp/terraform-exec#291
Reference: #992

This allows the acceptance testing framework to appropriately set the `TF_LOG` environment variable via `terraform-exec`, if Terraform is version 0.15 or later. Otherwise, defaults to `TRACE` similar to before when `TF_ACC_LOG_PATH` or `TF_LOG_PATH_MASK` environment variables are set.
bflad added a commit that referenced this issue Jul 1, 2022
…erraform during acceptance testing on versions 0.15 and later

Reference: hashicorp/terraform-exec#291
Reference: #992

This allows the acceptance testing framework to appropriately set the `TF_LOG` environment variable via `terraform-exec`, if Terraform is version 0.15 or later. Otherwise, defaults to `TRACE` similar to before when `TF_ACC_LOG_PATH` or `TF_LOG_PATH_MASK` environment variables are set.
@bflad
Copy link
Member

bflad commented Jul 1, 2022

Please check out #993 which should allow full control over the various Terraform CLI logging environment variables in the acceptance testing framework. That changeset also includes updates to the website documentation, which should hopefully describe the multitude of confusing logging environment variables better when running acceptance testing versus production usage.

@bflad bflad added the subsystem/tests Issues and feature requests related to the testing framework. label Jul 1, 2022
@bflad bflad closed this as completed in #993 Jul 5, 2022
bflad added a commit that referenced this issue Jul 5, 2022
…R` environment variable handling for Terraform versions 0.15 and later (#993)

Reference: #992

This enables provider developers to have full control over Terraform CLI logging environment variables with the acceptance testing framework in `helper/resource`.

In Terraform CLI 0.15 and later, the logging environment variables work as the following:

| Environment Variable | Description |
| - | - |
| `TF_LOG` | Global logging level for core and providers. Overrides `TF_LOG_CORE` (but not `TF_LOG_PROVIDER`) if set. |
| `TF_LOG_CORE` |  Logging level for core functionality. No effect if `TF_LOG` is set. |
| `TF_LOG_PROVIDER` | Logging level for provider logging sent over plugin file descriptors. |

To execute Terraform CLI during acceptance testing, it uses `terraform-exec` for all interactions. `terraform-exec` has certain environment variable handling rules associated with it, to prevent issues with reading stdout/stderr output from calling Terraform CLI since it needs to be able to parse that information. It was recently updated to support setting `TF_LOG`, `TF_LOG_CORE`, and `TF_LOG_PROVIDER` environment variables when calling Terraform, in addition to the prior support for setting `TF_LOG_PATH`, which defaults to always setting `TF_LOG` to `TRACE` if unset.

This enhancement allows the acceptance testing framework to accept the `TF_ACC_LOG` environment variable to appropriately set the `TF_LOG` environment variable in `terraform-exec`, if Terraform is version 0.15 or later. We could not change the behavior of the existing `TF_LOG` environment variable as it would cause undesirable side effects with the existing testing framework handling for the Go standard library `log` package. The `TF_ACC_LOG` naming follows the convention of `TF_ACC_LOG_PATH` setting `TF_LOG_PATH`. The `TF_LOG` value defaults to `TRACE`, similar to before, when `TF_ACC_LOG_PATH` or `TF_LOG_PATH_MASK` environment variables are set.

This also enables the passthrough of `TF_LOG_CORE` and `TF_LOG_PROVIDER` environment variables to `terraform-exec`. This is required because `terraform-exec` will otherwise default `TF_LOG` to `TRACE` if `TF_LOG_PATH` is used, meaning that without this, provider developers could not disable Terraform core logging. This enhancement opts to not prefix the variables with `TF_ACC_` as there's no prior behaviors to protect and the differing environment variable names for acceptance testing is already confusing enough for everyone. They could still be prefixed for consistency with `TF_ACC_LOG` though, if desired.

To summarize the state of logging environment variable with the acceptance testing framework after this change:

| Environment Variable | Description |
| - | - |
| `TF_ACC_LOG` | Controls the Terraform CLI `TF_LOG` environment variable |
| `TF_ACC_LOG_PATH` | Controls the Terraform CLI `TF_LOG_PATH` environment variable and where terraform-plugin-log SDK/provider loggers will write logs |
| `TF_LOG` | Controls the Go standard library `log` package log level; Any level enables terraform-plugin-log SDK/provider loggers at `TRACE` currently (they have to be tuned down individually at the moment) |
| `TF_LOG_CORE` | Controls the Terraform CLI `TF_LOG_CORE` environment variable |
| `TF_LOG_PATH_MASK` | Act similar to `TF_ACC_LOG_PATH`, but replaces a `%s` placeholder in the value with individual test names (`(testing.T).Name()`) |
| `TF_LOG_PROVIDER` | Controls the Terraform CLI `TF_LOG_PROVIDER` environment variable |
@github-actions
Copy link

github-actions bot commented Aug 5, 2022

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 5, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request subsystem/tests Issues and feature requests related to the testing framework.
Projects
None yet
2 participants