Change env_prefix at run-time

[whell]

Hi, is it possible to change env_prefix at runtime? I've tried some hacks which didn't work (playing with inheritance, overriding) but noting worked. Something like this would be nice:

import os
from typing import Dict, Tuple

from pydantic import BaseSettings, Extra
from pydantic.env_settings import SettingsSourceCallable


class SettingsConfigBase:
    env_file = os.environ.get("dotenv_config_path", ".env")
    env_prefix = ""  # This should be overriden by deriving classes
    case_sensitive = False
    extra = Extra.ignore

    @classmethod
    def customise_sources(
        cls,
        init_settings: SettingsSourceCallable,
        env_settings: SettingsSourceCallable,
        file_secret_settings: SettingsSourceCallable,
    ) -> Tuple[SettingsSourceCallable, ...]:
        # Prioritize environment settings over default config
        return env_settings, init_settings, file_secret_settings


class Settings(BaseSettings):
    foo: str = "foo_default"

    class Config(SettingsConfigBase):
        env_prefix = 'APREFIX_'


def main():
    # Something like this?
    settings = Settings(_env_prefix='BPREFIX_')
    print(settings.foo)


if __name__ == '__main__':
    main()

[whell]

I'll shamelessly self-bump this. Why I think being able to change env_prefix at runtime would be useful - this would allow to configure multiple instances of the same object differently.

[Null665]

I would like to put some effort into implementing this myself, if this has some potential to be included in a future versions of pydantic.
Looking through code, it does not look simple because Config model seems to be used when constructing the model by metaclass. Making env_prefix as a part EnvSettingsSource may be easier, if all edge cases are handled.

Any thoughts on this?

[jesusch]

I'd love/need that as well. I tried everything but nothing I came up with worked?

`
Settings.Config.env_prefix = 'bla_' # no

class Config:
env_prefix = 'bla_'

Setting.Config = Config
`

[ricardogsilva]

@jesusch @Null665 @whell

If anyone is also interested in this, I've found out that we need to do two things in order for it to work:

1. Override the env_prefix property of the class' __config__
2. Ask the __config__ instance to prepare fields again

I've put together a hack that works by doing something like this (yes, I know it is fugly):

import pydantic

class MySettings(pydantic.BaseSettings):
    param_a: bool


runtime_env_prefix = "something_set_at_runtime"

# 1. override the original env_prefix
MySettings.__config__.env_prefix = runtime_env_prefix

# 2. prepare fields again with the new env_prefix in place
for field in MySettings.__fields__.values():
    MySettings.__config__.prepare_field(field)

# now we can create our settings and it will look for env variables with the runtime env prefix
settings = MySettings()

[bgloss]

Tried to use it to access different sets of Postgres credentials from an .env-file, e.g., for admin and the db-owner -> No success

settings_admin = MySettings(_env_file=".env", _env_prefix="POSTGRES_ADMIN_")
settings_db_owner = MySettings(_env_file=".env", _env_prefix="POSTGRES_DB_OWNER_")

[dmontagu]

@ricardogsilva's solution should work if you use Config.env_prefix rather than the _env_prefix key. For what it's worth, I think we'll be more open to fixing these sorts of things once pydantic v2 is released.

[dmontagu]

To be clear, that would look something like this:

MySettings.__config__.env_prefix = "POSTGRES_ADMIN_"
for field in MySettings.__fields__.values():
    MySettings.__config__.prepare_field(field)
settings_admin = MySettings(_env_file=".env")

MySettings.__config__.env_prefix = "POSTGRES_DB_OWNER_"
for field in MySettings.__fields__.values():
    MySettings.__config__.prepare_field(field)
settings_db_owner = MySettings(_env_file=".env", )

I'll note that I have not tested this.

[bgloss]

Whoooo ... :-)
For the sake of simplicity of my code, I'll wait for v2 and use it in my next project.
This "configuration before init" is quite a thing!

[timonviola]

Running pydantic-settings 2.0.0 - I use inheritance to do this. For me it works better, as I don't need to worry about which .model_config changes I made in what order (changing this class variable is basically modifying global state).

import pydantic_settings

class FooLikeSettings(pydantic_settings.BaseSettings): 
   model_config = (
        pydantic_settings.SettingsConfigDict(
            env_prefix="foo_",
     )
     ... # model definition

class Bar(FooLikeSettings):
    pass

Bar.model_config["env_prefix"] = "bar_"    

Not sure if this is the right thing. Probably this should be discouraged in favor of creating nested models.