Skip to content

Stricter secret env vars: allow in profiles.yml + packages.yml, disallow elsewhere #4310

Closed
@jtcohen6

Description

@jtcohen6

In v0.21, we implemented "secret" env vars (prefixed with DBT_ENV_SECRET_) that enabled users to ensure that their configured credentials or access tokens would be scrubbed from logs.

While this is a convenience for many users, it's still possible to use those env vars in all the ways + places that {{ env_var() }} can be used in dbt projects—in arbitrary macros, query results, models that write to the database, etc.

After further discussion and deliberation, we've identified two tightly constrained use cases for "secret" env vars:

  • environment-based configuration of connection profiles (in profiles.yml)
  • git access tokens for private packages (in packages.yml)

As such, we should restrict use of secret-prefixed env vars, enabling them for use in those two files only, and raising an explicit error if they're used elsewhere. This enables project administrators to unblock their colleagues from connecting to the database and accessing dbt packages, without exposing those secrets to misuse.

Questions we're considering now, and will likely answer in the future:

  • Should this tight constraint be the default behavior? Should it be possible to re-enable more "permissive" use?
  • Are there other use cases we aren't capturing above? (As soon as secrets are allowed in dbt_project.yml or macros, they're effectively allowed everywhere)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Participants

      @jtcohen6

      Issue actions

        Stricter secret env vars: allow in `profiles.yml` + `packages.yml`, disallow elsewhere · Issue #4310 · dbt-labs/dbt-core