Misleading diagnostics "Attribute required" for attributes with defaults

[radeksimko]

Background

This is more of a meta issue describing a problem with "required-ness" of attributes, which was originally reported in

• Allow disabling syntax validation for blocks #1612
• provider block for github shows as invalid for an empty app_auth block #1597

Currently, we produce diagnostics for any attribute which is marked as "required" in the [JSON] provider schema. This is often correct in the simple cases, e.g. value in the output block:

However, it is incorrect/misleading in more complex cases, typically in provider blocks, which frequently get configured via environment variables:

The distinction between "just required" and "required if not supplied as environment variable" is currently embedded in the providers, as can be shown for the two cases above:

• https://github.com/hashicorp/terraform-provider-vault/blob/be64448357592b1f41a6f8b31fc8247c5ff63138/internal/provider/provider.go#L71-L72
• https://github.com/integrations/terraform-provider-github/blob/4a5ab5f5b08813c52cbf2f7b917aabd7fda623cd/github/provider.go#L76-L94

Unfortunately, this distinction (the existence of a possible default) is not surfaced in an accessible way - i.e. it's not part of the JSON machine-readable schema (terraform providers schema -json) we use to inform the validation.

Proposal

• Consider disabling validation of required attributes
• Consider Make individual validators configurable #1605 (perhaps with required attribute validator disabled by default for now)
• Plan what we can do to enable reliable validation (e.g. have providers expose the defaults somehow)

[dbanck]

We released a partial bug fix for provider blocks in 2.29.0.

I'm keeping this issue open in case anyone encounters this problem with other block types.

[ndrpnt]

Hi, facing the same issue with nutanix_karbon_cluster.storage_class_config.volumes_config.{username, password} attributes.