Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Integer-like strings in validator defaults are incorrectly cast to integers #585

Closed
JacobCallahan opened this issue May 11, 2021 · 7 comments
Assignees

Comments

@JacobCallahan
Copy link
Contributor

When specifying a default value for a validator of "+172800", dynaconf is casting the post-validation value to an integer 172800. This is still true after explicitly adding is_type_of=str to the validator.
Interestingly enough, performing another validation after this point fails the validation.

Validator section including the offending validator
Screenshot from 2021-05-11 10-53-25

Example of the problem
Screenshot from 2021-05-11 10-49-20

Dynaconf shouldn't be explicitly casting default values to another type when not explicitly conflicting with the is_type_of value.

Environment:

  • OS: Fedora: 33
  • Dynaconf Version: 3.1.4
  • Frameworks in use: None
@andressadotpy
Copy link
Collaborator

Hello @JacobCallahan! I'm trying to reproduce this bug here but I'm always receiving a string, even with those examples you added.

In [1]: from dynaconf.validator import Validator

In [2]: validator1 = Validator("Testing", is_type_of=str, default="+172800")

In [3]: validator1
Out[3]: <dynaconf.validator.Validator at 0x7fd4d6439160>

In [4]: dir(validator1)
Out[4]: 
['__and__',
 '__class__',
 '__delattr__',
 '__dict__',
 '__dir__',
 '__doc__',
 '__eq__',
 '__format__',
 '__ge__',
 '__getattribute__',
 '__gt__',
 '__hash__',
 '__init__',
 '__init_subclass__',
 '__le__',
 '__lt__',
 '__module__',
 '__ne__',
 '__new__',
 '__or__',
 '__reduce__',
 '__reduce_ex__',
 '__repr__',
 '__setattr__',
 '__sizeof__',
 '__str__',
 '__subclasshook__',
 '__weakref__',
 '_validate_items',
 'cast',
 'condition',
 'default',
 'default_messages',
 'description',
 'envs',
 'messages',
 'must_exist',
 'names',
 'operations',
 'validate',
 'when']

In [5]: validator1.default
Out[5]: '+172800'

In [6]: type(validator1.default)
Out[6]: str

In [7]: validator1 = Validator("Testing", is_type_of=int, default="+172800")

In [8]: type(validator1.default)
Out[8]: str

In [9]: validator1.default
Out[9]: '+172800'

In [10]: validator1 = Validator("Testing", is_type_of=int, default="3600")

In [11]: type(validator1.default)
Out[11]: str

In [12]: validator1 = Validator("Testing", default="3600")

In [13]: type(validator1.default)
Out[13]: str

In [14]: validator1 = Validator("Testing", default=3600)

In [15]: type(validator1.default)
Out[15]: int

Do you have an idea of how to reproduce this bug?

@JacobCallahan
Copy link
Contributor Author

@andressadotpy thanks for taking an interest in this issue!
So the problem isn't with the validator object itself, but when the default value is cast into a settings object.

In [1]: import dynaconf

In [2]: validator1 = dynaconf.validator.Validator("Testing", is_type_of=str, default="+172800")

In [3]: validator1.default
Out[3]: '+172800'

In [4]: settings = dynaconf.Dynaconf(validators=[validator1])

In [5]: settings.testing
Out[5]: 172800

In [6]: type(settings.testing)
Out[6]: int

@andressadotpy
Copy link
Collaborator

I'm having some trouble debugging the code for this issue, so I'll put some things here in hope someone can help me. I wrote a test for this:

def test_default_value_should_not_change_type_with_is_type_set():
    validator = Validator("TEST", is_type_of=str, default="+172800")
    settings = Dynaconf(validators=[validator])

    assert type(settings.validators) == ValidatorList
    assert settings.validators[0].default == "+172800"
    assert settings.test == "+172800"
================================== FAILURES ===================================
_________ test_default_value_should_not_change_type_with_is_type_set __________

    def test_default_value_should_not_change_type_with_is_type_set():
        validator = Validator("TEST", is_type_of=str, default="+172800")
        settings = Dynaconf(validators=[validator])
    
        assert type(settings.validators) == ValidatorList
        assert settings.validators[0].default == "+172800"
>       assert settings.test == "+172800"
E       AssertionError: assert 172800 == '+172800'
E         +172800
E         -'+172800'

/Users/andressa.cabistani/Documents/dev/dyn/dynaconf/tests/test_validators.py:670: AssertionError
=========================== short test summary info ===========================
FAILED tests/test_validators.py::test_default_value_should_not_change_type_with_is_type_set
======================== 1 failed, 34 passed in 1.15s =========================

By the assertions that fail, I found really curious that inside the settings.validators[0].default the value for default is a string. So I believe it's being changed in how Settings is delivering this, what do you think?

I used pdb inside __setattr__() for Settings and this is the output:

tests/test_validators.py::test_default_value_should_not_change_type_with_is_type_set 
>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(251)__setattr__()
-> super(Settings, self).__setattr__(name, value)
(Pdb) n
--Return--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(251)__setattr__()->None
-> super(Settings, self).__setattr__(name, value)
(Pdb) name
'validators'
(Pdb) value
[<dynaconf.validator.Validator object at 0x7fd6b13bfe10>]
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(226)__init__()
-> compat_kwargs(kwargs)
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(227)__init__()
-> if settings_module:
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(229)__init__()
-> for key, value in kwargs.items():
(Pdb) kwargs.items()
dict_items([])
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(232)__init__()
-> self._defaults = kwargs
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(233)__init__()
-> self.execute_loaders()
(Pdb) self._defaults
{}
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(235)__init__()
-> self.validators.validate(
(Pdb) self.validators
[<dynaconf.validator.Validator object at 0x7fd6b13bfe10>]
(Pdb) self.__dict__
{'_fresh': False, '_loaded_envs': [], '_loaded_hooks': defaultdict(<class 'dict'>, {}), '_loaded_py_modules': [], '_loaded_files': [], '_deleted': set(), '_store': <Box: {'RENAMED_VARS': {'DYNACONF_NAMESPACE': 'ENV_FOR_DYNACONF', 'NAMESPACE_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'DYNACONF_SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'DYNACONF_SETTINGS': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'PROJECT_ROOT': 'ROOT_PATH_FOR_DYNACONF', 'PROJECT_ROOT_FOR_DYNACONF': 'ROOT_PATH_FOR_DYNACONF', 'DYNACONF_SILENT_ERRORS': 'SILENT_ERRORS_FOR_DYNACONF', 'DYNACONF_ALWAYS_FRESH_VARS': 'FRESH_VARS_FOR_DYNACONF', 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT_ENV_FOR_DYNACONF', 'GLOBAL_ENV_FOR_DYNACONF': 'ENVVAR_PREFIX_FOR_DYNACONF'}, 'ROOT_PATH_FOR_DYNACONF': None, 'SETTINGS_FILE_FOR_DYNACONF': [], 'ENVIRONMENTS_FOR_DYNACONF': False, 'LOWERCASE_READ_FOR_DYNACONF': True, 'ENV_SWITCHER_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'ENV_FOR_DYNACONF': 'DEVELOPMENT', 'FORCE_ENV_FOR_DYNACONF': None, 'DEFAULT_ENV_FOR_DYNACONF': 'DEFAULT', 'ENVVAR_PREFIX_FOR_DYNACONF': 'DYNACONF', 'IGNORE_UNKNOWN_ENVVARS_FOR_DYNACONF': False, 'ENCODING_FOR_DYNACONF': 'utf-8', 'MERGE_ENABLED_FOR_DYNACONF': False, 'NESTED_SEPARATOR_FOR_DYNACONF': '__', 'ENVVAR_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'REDIS_FOR_DYNACONF': {'host': 'localhost', 'port': 6379, 'db': 0, 'decode_responses': True, 'username': None, 'password': None}, 'REDIS_ENABLED_FOR_DYNACONF': False, 'VAULT_FOR_DYNACONF': {'url': 'http://localhost:8200', 'token': None, 'cert': None, 'verify': None, 'timeout': None, 'proxies': None, 'allow_redirects': None}, 'VAULT_ENABLED_FOR_DYNACONF': False, 'VAULT_PATH_FOR_DYNACONF': 'dynaconf', 'VAULT_MOUNT_POINT_FOR_DYNACONF': 'secret', 'VAULT_ROOT_TOKEN_FOR_DYNACONF': None, 'VAULT_KV_VERSION_FOR_DYNACONF': 1, 'VAULT_AUTH_WITH_IAM_FOR_DYNACONF': False, 'VAULT_AUTH_ROLE_FOR_DYNACONF': None, 'VAULT_ROLE_ID_FOR_DYNACONF': None, 'VAULT_SECRET_ID_FOR_DYNACONF': None, 'CORE_LOADERS_FOR_DYNACONF': ['YAML', 'TOML', 'INI', 'JSON', 'PY'], 'LOADERS_FOR_DYNACONF': ['dynaconf.loaders.env_loader'], 'SILENT_ERRORS_FOR_DYNACONF': True, 'FRESH_VARS_FOR_DYNACONF': [], 'DOTENV_PATH_FOR_DYNACONF': None, 'DOTENV_VERBOSE_FOR_DYNACONF': False, 'DOTENV_OVERRIDE_FOR_DYNACONF': False, 'INSTANCE_FOR_DYNACONF': None, 'YAML_LOADER_FOR_DYNACONF': 'safe_load', 'COMMENTJSON_ENABLED_FOR_DYNACONF': False, 'SECRETS_FOR_DYNACONF': None, 'INCLUDES_FOR_DYNACONF': [], 'PRELOAD_FOR_DYNACONF': [], 'SKIP_FILES_FOR_DYNACONF': [], 'DYNACONF_NAMESPACE': 'DEVELOPMENT', 'NAMESPACE_FOR_DYNACONF': 'DEVELOPMENT', 'DYNACONF_SETTINGS_MODULE': [], 'DYNACONF_SETTINGS': [], 'SETTINGS_MODULE': [], 'SETTINGS_MODULE_FOR_DYNACONF': [], 'PROJECT_ROOT': None, 'PROJECT_ROOT_FOR_DYNACONF': None, 'DYNACONF_SILENT_ERRORS': True, 'DYNACONF_ALWAYS_FRESH_VARS': [], 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT', 'GLOBAL_ENV_FOR_DYNACONF': 'DYNACONF'}>, '_env_cache': {}, '_loaded_by_loaders': {}, '_loaders': ['dynaconf.loaders.env_loader'], '_defaults': {'RENAMED_VARS': <Box: {'DYNACONF_NAMESPACE': 'ENV_FOR_DYNACONF', 'NAMESPACE_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'DYNACONF_SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'DYNACONF_SETTINGS': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'PROJECT_ROOT': 'ROOT_PATH_FOR_DYNACONF', 'PROJECT_ROOT_FOR_DYNACONF': 'ROOT_PATH_FOR_DYNACONF', 'DYNACONF_SILENT_ERRORS': 'SILENT_ERRORS_FOR_DYNACONF', 'DYNACONF_ALWAYS_FRESH_VARS': 'FRESH_VARS_FOR_DYNACONF', 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT_ENV_FOR_DYNACONF', 'GLOBAL_ENV_FOR_DYNACONF': 'ENVVAR_PREFIX_FOR_DYNACONF'}>, 'ROOT_PATH_FOR_DYNACONF': None, 'SETTINGS_FILE_FOR_DYNACONF': [], 'ENVIRONMENTS_FOR_DYNACONF': False, 'LOWERCASE_READ_FOR_DYNACONF': True, 'ENV_SWITCHER_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'ENV_FOR_DYNACONF': 'DEVELOPMENT', 'FORCE_ENV_FOR_DYNACONF': None, 'DEFAULT_ENV_FOR_DYNACONF': 'DEFAULT', 'ENVVAR_PREFIX_FOR_DYNACONF': 'DYNACONF', 'IGNORE_UNKNOWN_ENVVARS_FOR_DYNACONF': False, 'ENCODING_FOR_DYNACONF': 'utf-8', 'MERGE_ENABLED_FOR_DYNACONF': False, 'NESTED_SEPARATOR_FOR_DYNACONF': '__', 'ENVVAR_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'REDIS_FOR_DYNACONF': <Box: {'host': 'localhost', 'port': 6379, 'db': 0, 'decode_responses': True, 'username': None, 'password': None}>, 'REDIS_ENABLED_FOR_DYNACONF': False, 'VAULT_FOR_DYNACONF': <Box: {'url': 'http://localhost:8200', 'token': None, 'cert': None, 'verify': None, 'timeout': None, 'proxies': None, 'allow_redirects': None}>, 'VAULT_ENABLED_FOR_DYNACONF': False, 'VAULT_PATH_FOR_DYNACONF': 'dynaconf', 'VAULT_MOUNT_POINT_FOR_DYNACONF': 'secret', 'VAULT_ROOT_TOKEN_FOR_DYNACONF': None, 'VAULT_KV_VERSION_FOR_DYNACONF': 1, 'VAULT_AUTH_WITH_IAM_FOR_DYNACONF': False, 'VAULT_AUTH_ROLE_FOR_DYNACONF': None, 'VAULT_ROLE_ID_FOR_DYNACONF': None, 'VAULT_SECRET_ID_FOR_DYNACONF': None, 'CORE_LOADERS_FOR_DYNACONF': ['YAML', 'TOML', 'INI', 'JSON', 'PY'], 'LOADERS_FOR_DYNACONF': ['dynaconf.loaders.env_loader'], 'SILENT_ERRORS_FOR_DYNACONF': True, 'FRESH_VARS_FOR_DYNACONF': [], 'DOTENV_PATH_FOR_DYNACONF': None, 'DOTENV_VERBOSE_FOR_DYNACONF': False, 'DOTENV_OVERRIDE_FOR_DYNACONF': False, 'INSTANCE_FOR_DYNACONF': None, 'YAML_LOADER_FOR_DYNACONF': 'safe_load', 'COMMENTJSON_ENABLED_FOR_DYNACONF': False, 'SECRETS_FOR_DYNACONF': None, 'INCLUDES_FOR_DYNACONF': [], 'PRELOAD_FOR_DYNACONF': [], 'SKIP_FILES_FOR_DYNACONF': [], 'DYNACONF_NAMESPACE': 'DEVELOPMENT', 'NAMESPACE_FOR_DYNACONF': 'DEVELOPMENT', 'DYNACONF_SETTINGS_MODULE': [], 'DYNACONF_SETTINGS': [], 'SETTINGS_MODULE': [], 'SETTINGS_MODULE_FOR_DYNACONF': [], 'PROJECT_ROOT': None, 'PROJECT_ROOT_FOR_DYNACONF': None, 'DYNACONF_SILENT_ERRORS': True, 'DYNACONF_ALWAYS_FRESH_VARS': [], 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT', 'GLOBAL_ENV_FOR_DYNACONF': 'DYNACONF'}, 'environ': environ({'TMPDIR': '/var/folders/zc/q_k75yl50t1bx91b3_fd8cb80000gn/T/', '__CFBundleIdentifier': 'com.apple.Terminal', 'XPC_FLAGS': '0x0', 'TERM': 'xterm-256color', 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.q2k9ZAvSJl/Listeners', 'XPC_SERVICE_NAME': '0', 'TERM_PROGRAM': 'Apple_Terminal', 'TERM_PROGRAM_VERSION': '440', 'TERM_SESSION_ID': '8FEE2EB6-8CB8-44CB-B36F-987944712D2F', 'SHELL': '/bin/zsh', 'HOME': '/Users/andressa.cabistani', 'LOGNAME': 'andressa.cabistani', 'USER': 'andressa.cabistani', 'PATH': '/Users/andressa.cabistani/opt/anaconda3/envs/dynaconf/bin:/Users/andressa.cabistani/opt/anaconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin', 'SHLVL': '1', 'PWD': '/Users/andressa.cabistani/Documents/dev/dyn/dynaconf', 'OLDPWD': '/Users/andressa.cabistani', 'ZSH': '/Users/andressa.cabistani/.oh-my-zsh', 'PAGER': 'less', 'LESS': '-R', 'LSCOLORS': 'Gxfxcxdxbxegedabagacad', 'CONDA_EXE': '/Users/andressa.cabistani/opt/anaconda3/bin/conda', '_CE_M': '', '_CE_CONDA': '', 'CONDA_PYTHON_EXE': '/Users/andressa.cabistani/opt/anaconda3/bin/python', 'CONDA_SHLVL': '2', 'CONDA_PREFIX': '/Users/andressa.cabistani/opt/anaconda3/envs/dynaconf', 'CONDA_DEFAULT_ENV': 'dynaconf', 'CONDA_PROMPT_MODIFIER': '', 'CONDA_PREFIX_1': '/Users/andressa.cabistani/opt/anaconda3', 'LC_CTYPE': 'UTF-8', '_': '/Users/andressa.cabistani/opt/anaconda3/envs/dynaconf/bin/pytest', 'PYTEST_CURRENT_TEST': 'tests/test_validators.py::test_default_value_should_not_change_type_with_is_type_set (call)'}), 'SETTINGS_MODULE': [], 'filter_strategy': None, '_not_installed_warnings': [], '_validate_only': None, '_validate_exclude': None, 'validators': [<dynaconf.validator.Validator object at 0x7fd6b13bfe10>], 'RENAMED_VARS': <Box: {'DYNACONF_NAMESPACE': 'ENV_FOR_DYNACONF', 'NAMESPACE_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'DYNACONF_SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'DYNACONF_SETTINGS': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE': 'SETTINGS_FILE_FOR_DYNACONF', 'SETTINGS_MODULE_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'PROJECT_ROOT': 'ROOT_PATH_FOR_DYNACONF', 'PROJECT_ROOT_FOR_DYNACONF': 'ROOT_PATH_FOR_DYNACONF', 'DYNACONF_SILENT_ERRORS': 'SILENT_ERRORS_FOR_DYNACONF', 'DYNACONF_ALWAYS_FRESH_VARS': 'FRESH_VARS_FOR_DYNACONF', 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT_ENV_FOR_DYNACONF', 'GLOBAL_ENV_FOR_DYNACONF': 'ENVVAR_PREFIX_FOR_DYNACONF'}>, 'ROOT_PATH_FOR_DYNACONF': None, 'SETTINGS_FILE_FOR_DYNACONF': [], 'ENVIRONMENTS_FOR_DYNACONF': False, 'LOWERCASE_READ_FOR_DYNACONF': True, 'ENV_SWITCHER_FOR_DYNACONF': 'ENV_FOR_DYNACONF', 'ENV_FOR_DYNACONF': 'DEVELOPMENT', 'FORCE_ENV_FOR_DYNACONF': None, 'DEFAULT_ENV_FOR_DYNACONF': 'DEFAULT', 'ENVVAR_PREFIX_FOR_DYNACONF': 'DYNACONF', 'IGNORE_UNKNOWN_ENVVARS_FOR_DYNACONF': False, 'ENCODING_FOR_DYNACONF': 'utf-8', 'MERGE_ENABLED_FOR_DYNACONF': False, 'NESTED_SEPARATOR_FOR_DYNACONF': '__', 'ENVVAR_FOR_DYNACONF': 'SETTINGS_FILE_FOR_DYNACONF', 'REDIS_FOR_DYNACONF': <Box: {'host': 'localhost', 'port': 6379, 'db': 0, 'decode_responses': True, 'username': None, 'password': None}>, 'REDIS_ENABLED_FOR_DYNACONF': False, 'VAULT_FOR_DYNACONF': <Box: {'url': 'http://localhost:8200', 'token': None, 'cert': None, 'verify': None, 'timeout': None, 'proxies': None, 'allow_redirects': None}>, 'VAULT_ENABLED_FOR_DYNACONF': False, 'VAULT_PATH_FOR_DYNACONF': 'dynaconf', 'VAULT_MOUNT_POINT_FOR_DYNACONF': 'secret', 'VAULT_ROOT_TOKEN_FOR_DYNACONF': None, 'VAULT_KV_VERSION_FOR_DYNACONF': 1, 'VAULT_AUTH_WITH_IAM_FOR_DYNACONF': False, 'VAULT_AUTH_ROLE_FOR_DYNACONF': None, 'VAULT_ROLE_ID_FOR_DYNACONF': None, 'VAULT_SECRET_ID_FOR_DYNACONF': None, 'CORE_LOADERS_FOR_DYNACONF': ['YAML', 'TOML', 'INI', 'JSON', 'PY'], 'LOADERS_FOR_DYNACONF': ['dynaconf.loaders.env_loader'], 'SILENT_ERRORS_FOR_DYNACONF': True, 'FRESH_VARS_FOR_DYNACONF': [], 'DOTENV_PATH_FOR_DYNACONF': None, 'DOTENV_VERBOSE_FOR_DYNACONF': False, 'DOTENV_OVERRIDE_FOR_DYNACONF': False, 'INSTANCE_FOR_DYNACONF': None, 'YAML_LOADER_FOR_DYNACONF': 'safe_load', 'COMMENTJSON_ENABLED_FOR_DYNACONF': False, 'SECRETS_FOR_DYNACONF': None, 'INCLUDES_FOR_DYNACONF': [], 'PRELOAD_FOR_DYNACONF': [], 'SKIP_FILES_FOR_DYNACONF': [], 'DYNACONF_NAMESPACE': 'DEVELOPMENT', 'NAMESPACE_FOR_DYNACONF': 'DEVELOPMENT', 'DYNACONF_SETTINGS_MODULE': [], 'DYNACONF_SETTINGS': [], 'SETTINGS_MODULE_FOR_DYNACONF': [], 'PROJECT_ROOT': None, 'PROJECT_ROOT_FOR_DYNACONF': None, 'DYNACONF_SILENT_ERRORS': True, 'DYNACONF_ALWAYS_FRESH_VARS': [], 'BASE_NAMESPACE_FOR_DYNACONF': 'DEFAULT', 'GLOBAL_ENV_FOR_DYNACONF': 'DYNACONF'}
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(236)__init__()
-> only=self._validate_only, exclude=self._validate_exclude
(Pdb) n
--Return--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(236)__init__()->None
-> only=self._validate_only, exclude=self._validate_exclude
(Pdb) n
--Call--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/utils/functional.py(43)__setattr__()
-> def __setattr__(self, name, value):
(Pdb) name
'_wrapped'
(Pdb) value
<dynaconf.base.Settings object at 0x7fd6b152ce48>
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/utils/functional.py(44)__setattr__()
-> if name in ["_wrapped", "_kwargs", "_warn_dynaconf_global_settings"]:
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/utils/functional.py(46)__setattr__()
-> self.__dict__[name] = value
(Pdb) name
'_wrapped'
(Pdb) value
<dynaconf.base.Settings object at 0x7fd6b152ce48>
(Pdb) n
--Return--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/utils/functional.py(46)__setattr__()->None
-> self.__dict__[name] = value
(Pdb) name
'_wrapped'
(Pdb) n
--Return--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(164)_setup()->None
-> settings_module=settings_module, **self._kwargs
(Pdb) self._kwargs
{'validators': [<dynaconf.validator.Validator object at 0x7fd6b13bfe10>]}
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(114)__getattr__()
-> if name in self._wrapped._deleted:  # noqa
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(119)__getattr__()
-> if name not in RESERVED_ATTRS:
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(128)__getattr__()
-> name.isupper()
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(136)__getattr__()
-> value = getattr(self._wrapped, name)
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(137)__getattr__()
-> if name not in RESERVED_ATTRS:
(Pdb) value
[<dynaconf.validator.Validator object at 0x7fd6b13bfe10>]
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(139)__getattr__()
-> return value
(Pdb) n
--Return--
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/dynaconf/base.py(139)__getattr__()->[<dynaconf.va...7fd6b13bfe10>]
-> return value
(Pdb) value
[<dynaconf.validator.Validator object at 0x7fd6b13bfe10>]
(Pdb) n
> /Users/andressa.cabistani/Documents/dev/dyn/dynaconf/tests/test_validators.py(670)test_default_value_should_not_change_type_with_is_type_set()
-> assert settings.validators[0].default == "+172800"
(Pdb) 

I tried investigating lots of places, but I didn't find out nothing really. I believe I miss something, maybe @rochacbruno has some idea where should I be checking?

@rochacbruno
Copy link
Member

@andressadotpy

The problem is on the line https://github.com/rochacbruno/dynaconf/blob/master/dynaconf/utils/parse_conf.py#L319

Every value is passed to parse_conf_data before settings to settings._store

then that value is going to parse_with_toml https://github.com/rochacbruno/dynaconf/blob/master/dynaconf/utils/parse_conf.py#L248

And when converting +d to toml, toml infers it as an integer.

In [4]: from dynaconf.utils.parse_conf import parse_with_toml

In [5]: type(parse_with_toml("+1234"))
Out[5]: int

In [6]: type(parse_with_toml("-1234"))
Out[6]: int

Possible solution is to have the value double quoted on the settings file "'+1234'"

In [7]: type(parse_with_toml("'+1234'"))
Out[7]: str

Or use one cast marker with @

In [7]: from dynaconf.utils.parse_conf import parse_conf_data

In [8]: parse_conf_data("+1234", tomlfy=True)
Out[8]: 1234

In [9]: type(parse_conf_data("+1234", tomlfy=True))
Out[9]: int

In [10]: type(parse_conf_data("@str +1234", tomlfy=True))
Out[10]: str

That would require us to add a note on documentation about this caveat https://www.dynaconf.com/envvars/#type-casting-and-lazy-values

Or

@andressadotpy @JacobCallahan we can make a decision and change the line

https://github.com/rochacbruno/dynaconf/blob/2fd232b88a7639503c8771077a7afaf0d875889e/dynaconf/utils/parse_conf.py#L319

to be

    if (
        isinstance(data, str) and
        data.startswith(("+", "-")) and
        data[1:].isdigit()
    ):
        # this makes signed integers to not be toml-parsed
        # ex: +1234 will be parsed as str("1234") not int(1234)
        return data

    # return parsed string value
    return _parse_conf_data(data, tomlfy=tomlfy, box_settings=box_settings)

@andressadotpy
Copy link
Collaborator

@rochacbruno I liked this solution, thanks for your help, sorry I couldn't help much! If @JacobCallahan agree with the changes, can I be assigned to this issue to make them? Thanks

@JacobCallahan
Copy link
Contributor Author

@rochacbruno @andressadotpy that should suit our case fine. Hopefully this is the only case (signed ints) where toml takes liberty in type conversion :)

@rochacbruno
Copy link
Member

@JacobCallahan actually TOML takes more integer prefixes in to account https://github.com/toml-lang/toml/blob/master/toml.abnf#L116-L129

We can resolve the case for + and - as suggested.

But in any case, @andressadotpy we need to update the docs adding some notes like

# toml conversions

All values in dynaconf are parsed using toml format, TOML tries to be smart
and infer the type of the settings variables, some variables will be automatically
converted to integer:


    FOO = "0x..."  # hexadecimal
    FOO = "0o..."  # Octal
    FOO = "0b..."  # Binary

All the cases are on toml specs https://github.com/toml-lang/toml/blob/master/toml.abnf

If you need to force a specific type casting there are 2 options.

1. Use double quoted for strings ex: `FOO = "'0x...'"  will be string.
2. Specify the type using `@`  ex: FOO = "@str 0x..."
   (available converters are `@int, @float, @bool, @json`)

andressadotpy pushed a commit to andressadotpy/dynaconf that referenced this issue Oct 7, 2021
…he string is a digit to unable toml to change data type of a string to an int to fix issue dynaconf#585; Update documentation about toml
rochacbruno pushed a commit that referenced this issue Oct 8, 2021
Co-authored-by: andressa.cabistani <andressa.cabistani@thoughtworks.com>
rochacbruno added a commit that referenced this issue Apr 15, 2022
Shortlog of commits since last release:

    Anderson Sousa (1):
          Document the usage with python -m (#710)

    Andressa Cabistani (2):
          Add unique label when merging lists to fix issue #653 (#661)
          Add new validation to fix issue #585 (#667)

    Armin Berres (1):
          Fix typo in error message

    Bruno Rocha (7):
          Release version 3.1.7
          Found this bug that was duplicating the generated envlist (#663)
          Add support for Python 3.10 (#665)
          Attempt to fix #555 (#669)
          Create update_contributors.yml
          Fixing pre-coomit and docs CI
          Added `dynaconf get` command to cli (#730)

    Caneco (2):
          improvement: add brand new logo to the project (#686)
          improvement: update socialcard to match the python way (#687)

    EdwardCuiPeacock (2):
          Feature: add @Jinja and @Format casting (#704)
          Combo converter doc (#735)

    Eitan Mosenkis (1):
          Fix FlaskConfig.setdefault (#706)

    Enderson Menezes (Mr. Enderson) (2):
          Force PYTHONIOENCODING to utf-8 to fix #664 (#672)
          edit: move discussions to github tab (#682)

    Eugene Triguba (1):
          Fix custom prefix link in envvar documentation (#680)

    Gibran Herrera (1):
          Fix Issue 662 Lazy validation (#675)

    Jitendra Yejare (2):
          Load vault secrets from environment less stores or which are not written by dynaconf (#725)
          Use default value when settings is blank (#729)

    Pavel Alimpiev (1):
          Update docs link (#678)

    Ugo Benassayag (1):
          Added validate_only_current_env to validator (issue #734) (#736)

    Waylon Walker (1):
          Docs Fix Spelling (#696)

    dependabot[bot] (3):
          Bump django from 2.1.5 to 2.2.26 in /example/django_pytest_pure (#711)
          Bump mkdocs from 1.1.2 to 1.2.3 (#715)
          Bump django from 2.2.26 to 2.2.27 in /example/django_pytest_pure (#717)

    github-actions[bot] (2):
          [automated] Update Contributors File (#691)
          [automated] Update Contributors File (#732)

    lowercase00 (1):
          Makes Django/Flask kwargs case insensitive (#721)
rochacbruno added a commit that referenced this issue Sep 22, 2022
Shortlog of commits since last release:

    Amadou Crookes (1):
          envars.md typo fix (#786)

    Bruno Rocha (19):
          Release version 3.1.9
          Bump dev version to 3.1.10
          Update badges
          demo repo will be replaced by a video tutorial soon
          Fix CI
          New data key casing must adapt to existing key casing (#795)
          Add test and docs about includes (#796)
          Removed vendor_src folder (#798)
          Replacing rochacbruno/ with dynaconf/ (#800)
          Fix codecov (#801)
          Parse negative numbers from envvar Fix #799 and Fix #585 (#802)
          Fix get command with Django (#804)
          Add a functional test runner (#805)
          Test runner docs and styling (#806)
          Allow merge_unique on lists when merge_enabled=True (#810)
          Rebind current env when forced for Pytest Fix #728 (#809)
          AUTO_CAST can be enabled on instance (#811)
          Ensure pyminify is on release script
          Add missing tomllib to monify script

    Gaurav Talreja (1):
          Fix #807 Use client.auth.approle.login instead of client.auth_approle (#808)

    Jitendra Yejare (1):
          Fix #768 of kv property depreciation from client object (#769)

    Joren Retel (2):
          Feature/detect casting comb token from converters (#784)
          Adding documentation and example to makefile. (#791)

    João Gustavo A. Amorim (1):
          Add pyupgrade hook (#759)

    Kian-Meng Ang (1):
          Fix typos (#788)

    Lucas Limeira (1):
          Using filter_strategy in env_loader to fix #760 (#767)

    Nicholas Nadeau, Ph.D., P.Eng (1):
          fix: typo (#766)

    Oleksii Baranov (2):
          Bump codecov action version (#775)
          Fix cli init command for flask (#705) (#774)

    Pedro de Medeiros (1):
          documentation fixes (#771)

    The Gitter Badger (1):
          Add a Gitter chat badge to README.md (#776)

    Théo Melo (1):
          Fixing a typo on the readme file (#763)

    Vicente Marçal (1):
          docs(pt-br): Docs Translation to brazilian portugues. (#787)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants