types: Remove Attrs, AttrTypes, Elems, ElemTypes, Null, Unknown, and …

…Value fields (#523)

Reference: #447
Reference: #502

When the framework type system was originally being developed, the value types were introduced with exported fields which also served as the internal details of whether a value was null, unknown, or a known value of a friendlier Go type. It was known that there was the potential for issues, but the simplified developer experience seemed to outweigh the potential for developer issues. Fast forward a few months, this decision appears to have two consequences that the framework maintainers hear about across various forums.

One issue is that the value types directly expose their internal implementation details and support the three states of a Terraform type value: being null, unknown, or a known value. Only one state should ever be set, but provider developers can make a value that is any combination of those states. This makes the framework behavior potentially indeterminate from the provider developer perspective whether, for example, a null AND unknown value becomes null OR unknown as it works its way through the framework.

```go
type ThingResourceModel struct{
  Computed types.String `tfsdk:"computed"`
}

func (r ThingResource) Create(ctx context.Context, req resource.CreateResource, resp *resource.CreateResponse) {
  var data ThingResourceModel

  resp.Diagnostics.Append(req.Plan.Get(ctx, &data)...)

  tflog.Trace(ctx, "Data Values", map[string]any{
    // Unknown value: types.String{Null: false, Unknown: true, Value: ""}
    "computed": plan.Computed,
  })

  // Maybe some external API responses here, but showing hardcoded updates for
  // brevity. This will make the value invalid by enabling Null without
  // disabling Unknown.
  data.Computed.Null = true

  tflog.Trace(ctx, "Data Values", map[string]any{
    // Invalid value: types.String{Null: true, Unknown: true, Value: ""}
    "computed": data.Computed,
  })

  // The invalid value will be either null or unknown, depending on the
  // type implementation. If unknown, Terraform will error, since unknown
  // values are never allowed in state.
  resp.Diagnostics.Append(resp.State.Set(ctx, &data)...)
}
```

Another issue is that it is possible to create collection value types that do not match their type definition. This issue is especially more likely with `types.Object` where it is possible to accidentially omit attributes. While the framework handling would eventually catch this issue when dealing with the invalid value, it can be caught sooner.

```go
// Invalid value (missing attribute and differing attribute name)
types.Object{
  AttrTypes: map[string]attr.Type{
    "one": types.StringType,
    "two": types.BoolType,
  },
  Attrs: map[string]attr.Value{
    "not_one": types.String{Value: "wrong name"},
  },
}
```

Another issue is that the default (zero-value) state for an "unset" value type turns into a known value, which is confusing since these values explicitly support being null. This causes Terraform errors which would surface to practitioners (especially when untested) that provider developers then have to troubleshoot the error message containing Terraform's type system details, potentially discover the reason why it is happening by looking at the framework type source code, then figure out a workable solution. It's not intuitive.

```go
type ThingResourceModel struct{
  // let's assume this is left unconfigured (null in config and plan)
  Optional types.String `tfsdk:"optional"`
}

func (r ThingResource) Create(ctx context.Context, req resource.CreateResource, resp *resource.CreateResponse) {
  // Providers can opt to use a single variable that is updated based on an
  // external response, however that logic can be more difficult sometimes,
  // so it can be easier to split them. Showing the split way to exemplify
  // the "unset" problem.
  var plan, state ThingResourceModel

  resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)

  tflog.Trace(ctx, "Plan Values", map[string]any{
    // Null value: types.String{Null: true, Unknown: false, Value: ""}
    "optional": plan.Optional,
  })

  // Maybe some external API responses here, but intentionally not
  // doing any state.Optional setting, which might happen if the
  // external response for that data was null for example.

  tflog.Trace(ctx, "State Values", map[string]any{
    // Zero-value: types.String{Null: false, Unknown: false, Value: ""}
    "optional": state.Optional,
  })

  // The state zero-value will later cause Terraform to error, such as:
  // Error: Provider produced inconsistent result after apply
  // ... expected cty.NullVal(cty.String), got cty.StringVal("")
  // Since the plan value said it would be null.
  resp.Diagnostics.Append(resp.State.Set(ctx, &state)...)
}
```

This removal of the fields in preference of functions and methods aims to unexport the internal details and treat the value types as immutable once they are created. The zero-value implementations of these values is now null, instead of a known zero-value of the underlying type.

While not recommended for production usage without extensive testing, provider developers can opt for `panic()` inducing versions of collection value creations, rather than dealing with `diag.Diagnostics` everywhere.

Accessing value information after the migration can be accomplished with the following:

| Prior Value Access | New Value Access |
|--------------------|------------------|
| `(types.Bool).Value` | `(types.Bool).ValueBool()` |
| `(types.Bool).Null` | `(types.Bool).IsNull()` |
| `(types.Bool).Unknown` | `(types.Bool).IsUnknown()` |
| `(types.Float64).Value` | `(types.Float64).ValueFloat64()` |
| `(types.Float64).Null` | `(types.Float64).IsNull()` |
| `(types.Float64).Unknown` | `(types.Float64).IsUnknown()` |
| `(types.Int64).Value` | `(types.Int64).ValueInt64()` |
| `(types.Int64).Null` | `(types.Int64).IsNull()` |
| `(types.Int64).Unknown` | `(types.Int64).IsUnknown()` |
| `(types.List).Elems` | `(types.List).Elements()` or `(types.List).ElementsAs()` |
| `(types.List).ElemType` | `(types.List).ElementType()` |
| `(types.List).Null` | `(types.List).IsNull()` |
| `(types.List).Unknown` | `(types.List).IsUnknown()` |
| `(types.Map).Elems` | `(types.Map).Elements()` or `(types.Map).ElementsAs()` |
| `(types.Map).ElemType` | `(types.Map).ElementType()` |
| `(types.Map).Null` | `(types.Map).IsNull()` |
| `(types.Map).Unknown` | `(types.Map).IsUnknown()` |
| `(types.Number).Value` | `(types.Number).ValueBigFloat()` |
| `(types.Number).Null` | `(types.Number).IsNull()` |
| `(types.Number).Unknown` | `(types.Number).IsUnknown()` |
| `(types.Object).Attrs` | `(types.Object).Attributes()` or `(types.Object).As()` |
| `(types.Object).AttrTypes` | `(types.Object).AttributeTypes()` |
| `(types.Object).Null` | `(types.Object).IsNull()` |
| `(types.Object).Unknown` | `(types.Object).IsUnknown()` |
| `(types.Set).Elems` | `(types.Set).Elements()` or `(types.Set).ElementsAs()` |
| `(types.Set).ElemType` | `(types.Set).ElementType()` |
| `(types.Set).Null` | `(types.Set).IsNull()` |
| `(types.Set).Unknown` | `(types.Set).IsUnknown()` |
| `(types.String).Value` | `(types.String).ValueString()` |
| `(types.String).Null` | `(types.String).IsNull()` |
| `(types.String).Unknown` | `(types.String).IsUnknown()` |

Creating values after the migration can be accomplished with the following:

| Prior Value Creation | New Value Creation |
|----------------------|--------------------|
| `types.Bool{Value: /* value */}` | `types.BoolValue(/* value */)` |
| `types.Bool{Null: true}` | `types.BoolNull()` |
| `types.Bool{Unknown: true}` | `types.BoolUnknown()` |
| `types.Float64{Value: /* value */}` | `types.Float64Value(/* value */)` |
| `types.Float64{Null: true}` | `types.Float64Null()` |
| `types.Float64{Unknown: true}` | `types.Float64Unknown()` |
| `types.Int64{Value: /* value */}` | `types.Int64Value(/* value */)` |
| `types.Int64{Null: true}` | `types.Int64Null()` |
| `types.Int64{Unknown: true}` | `types.Int64Unknown()` |
| `types.List{ElemType: /* element type */, Elems: /* value */}` | `list, diags := types.ListValue(/* element type */, /* value */)` or `list, diags := types.ListValueFrom(context.Context, /* element type */, any)` or `list := types.ListValueMust(/* element type */, /* value */)` |
| `types.List{ElemType: /* element type */, Null: true}` | `types.ListNull(/* element type */)` |
| `types.List{ElemType: /* element type */, Unknown: true}` | `types.ListUnknown(/* element type */)` |
| `types.Map{ElemType: /* element type */, Elems: /* value */}` | `m, diags := types.MapValue(/* element type */, /* value */)` or `m, diags := types.MapValueFrom(context.Context, /* element type */, any)` or `m := types.MapValueMust(/* element type */, /* value */)` |
| `types.Map{ElemType: /* element type */, Null: true}` | `types.MapNull(/* element type */)` |
| `types.Map{ElemType: /* element type */, Unknown: true}` | `types.MapUnknown(/* element type */)` |
| `types.Number{Value: /* value */}` | `types.NumberValue(/* value */)` |
| `types.Number{Null: true}` | `types.NumberNull()` |
| `types.Number{Unknown: true}` | `types.NumberUnknown()` |
| `types.Object{AttrTypes: /* attribute types */, Attrs: /* attribute values */}` | `object, diags := types.ObjectValue(/* attribute types */, /* attribute values */)` or `object, diags := types.ObjectValueFrom(context.Context, /* attribute types */, any)` or `object := types.ObjectValueMust(/* attribute types */, /* attribute values */)` |
| `types.Object{AttrTypes: /* attribute types */, Null: true}` | `types.ObjectNull(/* attribute types */)` |
| `types.Object{AttrTypes: /* attribute types */, Unknown: true}` | `types.ObjectUnknown(/* attribute types */)` |
| `types.Set{ElemType: /* element type */, Elems: /* value */}` | `set, diags := types.SetValue(/* element type */, /* value */)` or `set, diags := types.SetValueFrom(context.Context, /* element type */, any)` or `set := types.SetValueMust(/* element type */, /* value */)` |
| `types.Set{ElemType: /* element type */, Null: true}` | `types.SetNull(/* element type */)` |
| `types.Set{ElemType: /* element type */, Unknown: true}` | `types.SetUnknown(/* element type */)` |
| `types.String{Value: /* value */}` | `types.StringValue(/* value */)` |
| `types.String{Null: true}` | `types.StringNull()` |
| `types.String{Unknown: true}` | `types.StringUnknown()` |

.changelog/523.txt

@@ -0,0 +1,47 @@
+```release-note:breaking-change
+types: The `Bool` type `Null`, `Unknown`, and `Value` fields have been removed. Use the `BoolNull()`, `BoolUnknown()`, and `BoolValue()` creation functions and `IsNull()`, `IsUnknown()`, and `ValueBool()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Float64` type `Null`, `Unknown`, and `Value` fields have been removed. Use the `Float64Null()`, `Float64Unknown()`, and `Float64Value()` creation functions and `IsNull()`, `IsUnknown()`, and `ValueFloat64()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Int64` type `Null`, `Unknown`, and `Value` fields have been removed. Use the `Int64Null()`, `Int64Unknown()`, and `Int64Value()` creation functions and `IsNull()`, `IsUnknown()`, and `ValueInt64()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `List` type `Elems`, `ElemType`, `Null`, and `Unknown` fields have been removed. Use the `ListNull()`, `ListUnknown()`, `ListValue()`, and `ListValueMust()` creation functions and `Elements()`, `ElementsAs()`, `ElementType()`, `IsNull()`, and `IsUnknown()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Map` type `Elems`, `ElemType`, `Null`, and `Unknown` fields have been removed. Use the `MapNull()`, `MapUnknown()`, `MapValue()`, and `MapValueMust()` creation functions and `Elements()`, `ElementsAs()`, `ElementType()`, `IsNull()`, and `IsUnknown()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Number` type `Null`, `Unknown`, and `Value` fields have been removed. Use the `NumberNull()`, `NumberUnknown()`, and `NumberValue()` creation functions and `IsNull()`, `IsUnknown()`, and `ValueBigFloat()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Object` type `Attrs`, `AttrTypes`, `Null`, and `Unknown` fields have been removed. Use the `ObjectNull()`, `ObjectUnknown()`, `ObjectValue()`, and `ObjectValueMust()` creation functions and `As()`, `Attributes()`, `AttributeTypes()`, `IsNull()`, and `IsUnknown()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `Set` type `Elems`, `ElemType`, `Null`, and `Unknown` fields have been removed. Use the `SetNull()`, `SetUnknown()`, `SetValue()`, and `SetValueMust()` creation functions and `Elements()`, `ElementsAs()`, `ElementType()`, `IsNull()`, and `IsUnknown()` methods instead.
+```
+
+```release-note:breaking-change
+types: The `String` type `Null`, `Unknown`, and `Value` fields have been removed. Use the `StringNull()`, `StringUnknown()`, and `StringValue()` creation functions and `IsNull()`, `IsUnknown()`, and `ValueString()` methods instead.
+```
+
+```release-note:enhancement
+attr: Added `ValueState` type, which custom types can use to consistently represent the three possible value states (known, null, and unknown)
+```
+
+```release-note:bug
+types: Prevented Terraform errors where the zero-value for any `attr.Value` types such as `String` would be a known value instead of null
+```
+
+```release-note:bug
+types: Prevented indeterminate behavior for any `attr.Value` types where they could be any combination of null, unknown, and/or known
+```

attr/value_state.go

@@ -0,0 +1,31 @@
+package attr
+
+import "fmt"
+
+const (
+	// ValueStateNull represents a value which is null.
+	//
+	// This value is 0 so it is the zero-value for types implementations.
+	ValueStateNull ValueState = 0
+
+	// ValueStateUnknown represents a value which is unknown.
+	ValueStateUnknown ValueState = 1
+
+	// ValueStateKnown represents a value which is known (not null or unknown).
+	ValueStateKnown ValueState = 2
+)
+
+type ValueState uint8
+
+func (s ValueState) String() string {
+	switch s {
+	case ValueStateKnown:
+		return "known"
+	case ValueStateNull:
+		return "null"
+	case ValueStateUnknown:
+		return "unknown"
+	default:
+		panic(fmt.Sprintf("unhandled ValueState in String: %d", s))
+	}
+}

internal/fromtftypes/attribute_path_step_test.go

@@ -43,7 +43,7 @@ func TestAttributePathStep(t *testing.T) {
 		"PathStepElementKeyValue": {
 			tfType:   tftypes.ElementKeyValue(tftypes.NewValue(tftypes.String, "test")),
 			attrType: types.StringType,
-			expected: path.PathStepElementKeyValue{Value: types.String{Value: "test"}},
+			expected: path.PathStepElementKeyValue{Value: types.StringValue("test")},
 		},
 		"PathStepElementKeyValue-error": {
 			tfType:        tftypes.ElementKeyValue(tftypes.NewValue(tftypes.String, "test")),

internal/fromtftypes/attribute_path_test.go

@@ -88,7 +88,7 @@ func TestAttributePath(t *testing.T) {
 					},
 				},
 			},
-			expected: path.Root("test").AtSetValue(types.String{Value: "test-value"}),
+			expected: path.Root("test").AtSetValue(types.StringValue("test-value")),
 		},
 		"AttributeName-ElementKeyValue-value-conversion-error": {
 			tfType: tftypes.NewAttributePath().WithAttributeName("test").WithElementKeyValue(tftypes.NewValue(tftypes.String, "test-value")),

internal/fromtftypes/value_test.go

@@ -27,7 +27,7 @@ func TestValue(t *testing.T) {
 		"empty-tftype": {
 			tfType:   tftypes.Value{},
 			attrType: types.BoolType,
-			expected: types.Bool{Null: true},
+			expected: types.BoolNull(),
 		},
 		"nil-attr-type": {
 			tfType:        tftypes.Value{},
@@ -44,47 +44,47 @@ func TestValue(t *testing.T) {
 		"bool-null": {
 			tfType:   tftypes.NewValue(tftypes.Bool, nil),
 			attrType: types.BoolType,
-			expected: types.Bool{Null: true},
+			expected: types.BoolNull(),
 		},
 		"bool-unknown": {
 			tfType:   tftypes.NewValue(tftypes.Bool, tftypes.UnknownValue),
 			attrType: types.BoolType,
-			expected: types.Bool{Unknown: true},
+			expected: types.BoolUnknown(),
 		},
 		"bool-value": {
 			tfType:   tftypes.NewValue(tftypes.Bool, true),
 			attrType: types.BoolType,
-			expected: types.Bool{Value: true},
+			expected: types.BoolValue(true),
 		},
 		"float64-null": {
 			tfType:   tftypes.NewValue(tftypes.Number, nil),
 			attrType: types.Float64Type,
-			expected: types.Float64{Null: true},
+			expected: types.Float64Null(),
 		},
 		"float64-unknown": {
 			tfType:   tftypes.NewValue(tftypes.Number, tftypes.UnknownValue),
 			attrType: types.Float64Type,
-			expected: types.Float64{Unknown: true},
+			expected: types.Float64Unknown(),
 		},
 		"float64-value": {
 			tfType:   tftypes.NewValue(tftypes.Number, big.NewFloat(1.2)),
 			attrType: types.Float64Type,
-			expected: types.Float64{Value: 1.2},
+			expected: types.Float64Value(1.2),
 		},
 		"int64-null": {
 			tfType:   tftypes.NewValue(tftypes.Number, nil),
 			attrType: types.Int64Type,
-			expected: types.Int64{Null: true},
+			expected: types.Int64Null(),
 		},
 		"int64-unknown": {
 			tfType:   tftypes.NewValue(tftypes.Number, tftypes.UnknownValue),
 			attrType: types.Int64Type,
-			expected: types.Int64{Unknown: true},
+			expected: types.Int64Unknown(),
 		},
 		"int64-value": {
 			tfType:   tftypes.NewValue(tftypes.Number, 123),
 			attrType: types.Int64Type,
-			expected: types.Int64{Value: 123},
+			expected: types.Int64Value(123),
 		},
 		"list-null": {
 			tfType: tftypes.NewValue(
@@ -96,10 +96,7 @@ func TestValue(t *testing.T) {
 			attrType: types.ListType{
 				ElemType: types.StringType,
 			},
-			expected: types.List{
-				ElemType: types.StringType,
-				Null:     true,
-			},
+			expected: types.ListNull(types.StringType),
 		},
 		"list-unknown": {
 			tfType: tftypes.NewValue(
@@ -111,10 +108,7 @@ func TestValue(t *testing.T) {
 			attrType: types.ListType{
 				ElemType: types.StringType,
 			},
-			expected: types.List{
-				ElemType: types.StringType,
-				Unknown:  true,
-			},
+			expected: types.ListUnknown(types.StringType),
 		},
 		"list-value": {
 			tfType: tftypes.NewValue(
@@ -128,27 +122,27 @@ func TestValue(t *testing.T) {
 			attrType: types.ListType{
 				ElemType: types.StringType,
 			},
-			expected: types.List{
-				ElemType: types.StringType,
-				Elems: []attr.Value{
-					types.String{Value: "test-value"},
+			expected: types.ListValueMust(
+				types.StringType,
+				[]attr.Value{
+					types.StringValue("test-value"),
 				},
-			},
+			),
 		},
 		"number-null": {
 			tfType:   tftypes.NewValue(tftypes.Number, nil),
 			attrType: types.NumberType,
-			expected: types.Number{Null: true},
+			expected: types.NumberNull(),
 		},
 		"number-unknown": {
 			tfType:   tftypes.NewValue(tftypes.Number, tftypes.UnknownValue),
 			attrType: types.NumberType,
-			expected: types.Number{Unknown: true},
+			expected: types.NumberUnknown(),
 		},
 		"number-value": {
 			tfType:   tftypes.NewValue(tftypes.Number, big.NewFloat(1.2)),
 			attrType: types.NumberType,
-			expected: types.Number{Value: big.NewFloat(1.2)},
+			expected: types.NumberValue(big.NewFloat(1.2)),
 		},
 		"object-null": {
 			tfType: tftypes.NewValue(
@@ -164,12 +158,11 @@ func TestValue(t *testing.T) {
 					"test_attr": types.StringType,
 				},
 			},
-			expected: types.Object{
-				AttrTypes: map[string]attr.Type{
+			expected: types.ObjectNull(
+				map[string]attr.Type{
 					"test_attr": types.StringType,
 				},
-				Null: true,
-			},
+			),
 		},
 		"object-unknown": {
 			tfType: tftypes.NewValue(
@@ -185,12 +178,11 @@ func TestValue(t *testing.T) {
 					"test_attr": types.StringType,
 				},
 			},
-			expected: types.Object{
-				AttrTypes: map[string]attr.Type{
+			expected: types.ObjectUnknown(
+				map[string]attr.Type{
 					"test_attr": types.StringType,
 				},
-				Unknown: true,
-			},
+			),
 		},
 		"object-value": {
 			tfType: tftypes.NewValue(
@@ -208,14 +200,14 @@ func TestValue(t *testing.T) {
 					"test_attr": types.StringType,
 				},
 			},
-			expected: types.Object{
-				AttrTypes: map[string]attr.Type{
+			expected: types.ObjectValueMust(
+				map[string]attr.Type{
 					"test_attr": types.StringType,
 				},
-				Attrs: map[string]attr.Value{
-					"test_attr": types.String{Value: "test-value"},
+				map[string]attr.Value{
+					"test_attr": types.StringValue("test-value"),
 				},
-			},
+			),
 		},
 		"set-null": {
 			tfType: tftypes.NewValue(
@@ -227,10 +219,7 @@ func TestValue(t *testing.T) {
 			attrType: types.SetType{
 				ElemType: types.StringType,
 			},
-			expected: types.Set{
-				ElemType: types.StringType,
-				Null:     true,
-			},
+			expected: types.SetNull(types.StringType),
 		},
 		"set-unknown": {
 			tfType: tftypes.NewValue(
@@ -242,10 +231,7 @@ func TestValue(t *testing.T) {
 			attrType: types.SetType{
 				ElemType: types.StringType,
 			},
-			expected: types.Set{
-				ElemType: types.StringType,
-				Unknown:  true,
-			},
+			expected: types.SetUnknown(types.StringType),
 		},
 		"set-value": {
 			tfType: tftypes.NewValue(
@@ -259,27 +245,27 @@ func TestValue(t *testing.T) {
 			attrType: types.SetType{
 				ElemType: types.StringType,
 			},
-			expected: types.Set{
-				ElemType: types.StringType,
-				Elems: []attr.Value{
-					types.String{Value: "test-value"},
+			expected: types.SetValueMust(
+				types.StringType,
+				[]attr.Value{
+					types.StringValue("test-value"),
 				},
-			},
+			),
 		},
 		"string-null": {
 			tfType:   tftypes.NewValue(tftypes.String, nil),
 			attrType: types.StringType,
-			expected: types.String{Null: true},
+			expected: types.StringNull(),
 		},
 		"string-unknown": {
 			tfType:   tftypes.NewValue(tftypes.String, tftypes.UnknownValue),
 			attrType: types.StringType,
-			expected: types.String{Unknown: true},
+			expected: types.StringUnknown(),
 		},
 		"string-value": {
 			tfType:   tftypes.NewValue(tftypes.String, "test-value"),
 			attrType: types.StringType,
-			expected: types.String{Value: "test-value"},
+			expected: types.StringValue("test-value"),
 		},
 	}