Add Logging for Differences Between Proposed New State and Prior State During Planning

[bflad]

Module version

v1.1.1

Use-cases

During the PlanResourceChange RPC, the framework currently performs the following logic:

• If there are differences between the proposed new state and prior state, automatically mark any Computed attributes without a configuration value as unknown.
• Run schema-based plan modification
• Run resource-based plan modification

The first piece of logic there is to prevent a large class of Terraform inconsistent result after apply errors for practitioners where Terraform requires that an applied state matches the planned state. The tradeoff here is that it is up to provider developers to "enhance" the plan to be more accurate by converting those unknown values if they are not expected to change (e.g. UseStateForUnknown()).

The proposed new state and prior state come from Terraform across the protocol. (Aside: We currently do nothing to this data except normalize empty collection block values to null, however this is done for all data coming across so it should never introduce unintentional differences between them.) Essentially, we are trusting that Terraform is correctly calculating the proposed new state data, but we have found a few cases where Terraform was not doing that perfectly given the framework's more robust type system as compared to the prior SDK.

Provider developers have raised a few issues around this. The rub here is that they are seeing the symptoms of these issues where Terraform is showing unexpected plans with these unknown values because of the framework automatically modifying the plan instead of the root causes that triggered the framework's automatic plan modification. Sometimes this can lead to wasted development time looking at the attributes marked as unknown, rather than the real issue.

Attempted Solutions

Troubleshooting these situations is more difficult than it should be, especially in the case of provider acceptance testing. Current options (if you know exactly for what you are looking for) right now are:

• Using the TF_LOG_SDK_PROTO_DATA_DIR environment variable to inspect the PlanResourceChange RPC proposed new state and prior state data.
• Inspecting the JSON plan
• (Sometimes) Comparing states after re-applies

Proposal

When the framework is detecting a difference between the proposed new state and prior state, it could start logging the attribute paths of which root attributes/blocks are unequal. For example:

TIMESTAMP [DEBUG] sdk.framework: Detected value change between proposed new state and prior state: tf_attribute_path=attr1 ... other existing fields ...

For very safe cases, such as null <=> "" (hashicorp/terraform#31887), we could consider promoting that to WARN as its a known issue and potentially tailor the message further. We should avoid showing known values outside of Go zero-values in this logging though, as those known values may contain sensitive data which would be very complex to filter/mask.

References

• Plan Modifiers Not Executed Before Plan Computed-Only Unknown Marking #183
• Adding a computed SingleNestedAttribute to a resource causes other fields show (known after apply) #599
• Two computed values on a resource with one planmodifier results in unexpected new value #615
• computed field producing spurious plan changes with framework but not SDK v2 #628
• CLI Plan Output Not Showing null to "" Differences for Non-Legacy Type System Providers terraform#31887
• Dirty plan when schema contains a nested computed attribute that also has a computed attribute. terraform#32522

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.