diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index f2bda872..2c31930a 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -33,5 +33,18 @@ jobs: python -m pip install --upgrade pip pip install tox tox-gh-actions + - name: Check external links in the package documentation + run: tox -e linkcheck + - name: Build and test package documentation run: tox -e docs + + - name: Archive docs artifacts + if: always() + uses: actions/upload-artifact@v2 + with: + name: docs + path: docs + # Artifacts are retained for 90 days by default. + # In fact, we don't need such long period. + retention-days: 60 diff --git a/AUTHORS.rst b/AUTHORS.rst index ab55177e..6bf39824 100644 --- a/AUTHORS.rst +++ b/AUTHORS.rst @@ -6,6 +6,9 @@ and currently maintained by `Serghei Iakovlev `_ A full list of contributors can be found in `GitHub `__. +Acknowledgments +=============== + The existence of ``django-environ`` would have been impossible without these projects: diff --git a/BACKERS.rst b/BACKERS.rst index aa2a8148..4ac3b73c 100644 --- a/BACKERS.rst +++ b/BACKERS.rst @@ -9,9 +9,9 @@ Sponsors -------- Support this project by becoming a sponsor. Your logo will show up here with a -link to your website. `Became sponsor `_. +link to your website. `Became sponsor `_. -|ocsponsor0| |ocsponsor1| |ocsponsor2| +|ocsponsor0| |ocsponsor1| Backers ------- @@ -21,14 +21,11 @@ Thank you to all our backers! |ocbackerimage| .. |ocsponsor0| image:: https://opencollective.com/django-environ/sponsor/0/avatar.svg - :target: https://opencollective.com/django-environ/sponsor/0/website - :alt: Sponsor -.. |ocsponsor1| image:: https://opencollective.com/django-environ/sponsor/1/avatar.svg - :target: https://opencollective.com/django-environ/sponsor/1/website - :alt: Sponsor -.. |ocsponsor2| image:: https://opencollective.com/django-environ/sponsor/2/avatar.svg - :target: https://opencollective.com/django-environ/sponsor/2/website + :target: https://triplebyte.com/ :alt: Sponsor +.. |ocsponsor1| image:: https://images.opencollective.com/static/images/become_sponsor.svg + :target: https://opencollective.com/django-environ/contribute/sponsors-3474/checkout + :alt: Become a Sponsor .. |ocbackerimage| image:: https://opencollective.com/django-environ/backers.svg?width=890 :target: https://opencollective.com/django-environ :alt: Backers on Open Collective diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 0e55de02..3b3f5576 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,74 +1,106 @@ Changelog ========= + All notable changes to this project will be documented in this file. +The format is inspired by `Keep a Changelog `_ +and this project adheres to `Semantic Versioning `_. + +`v0.7.0`_ - 11-September-2021 +------------------------------ +Added ++++++ +- Added support for negative float strings + `#160 `_. +- Added Elasticsearch5 to search scheme + `#297 `_. +- Added Elasticsearch7 to search scheme + `#314 `_. +- Added the ability to use ``bytes`` or ``str`` as a default value for ``Env.bytes()``. + +Fixed ++++++ +- Fixed links in the documentation. +- Use default option in ``Env.bytes()`` + `#206 `_. +- Safely evaluate a string containing an invalid Python literal + `#200 `_. + +Changed ++++++++ +- Added 'Funding' and 'Say Thanks!' project urls on pypi. +- Stop raising ``UserWarning`` if ``.env`` file isn't found. Log a message with + ``INFO`` log level instead `#243 `_. -The format is *inspired* by `Keep a Changelog `_ -and this project adheres to `Semantic Versioning `_. `v0.6.0`_ - 4-September-2021 ---------------------------- Added +++++ - - Python 3.9, 3.10 and pypy 3.7 are now supported - - Django 3.1 and 3.2 are now supported - - Added missed classifiers to ``setup.py`` - - Accept Python 3.6 path-like objects for ``read_env`` +- Python 3.9, 3.10 and pypy 3.7 are now supported. +- Django 3.1 and 3.2 are now supported. +- Added missed classifiers to ``setup.py``. +- Accept Python 3.6 path-like objects for ``read_env`` + `#106 `_, + `#286 `_. Fixed +++++ - - Fixed various code linting errors - - Fixed typos in the documentation - - Added missed files to the package contents - - Fixed ``db_url_config`` to work the same for all postgres-like schemes +- Fixed various code linting errors. +- Fixed typos in the documentation. +- Added missed files to the package contents. +- Fixed ``db_url_config`` to work the same for all postgres-like schemes + `#264 `_, + `#268 `_. Changed +++++++ - - Refactor tests to use pytest and follow DRY - - Moved CI to GitHub Actions - - Restructuring of project documentation - - Build and test package documentation as a part of CI pipeline - - Build and test package distribution as a part of CI pipeline - - Check ``MANIFEST.in`` in a source package for completeness as a part of CI pipeline - - Added ``pytest`` and ``coverage[toml]`` to setuptools' ``extras_require`` +- Refactor tests to use pytest and follow DRY. +- Moved CI to GitHub Actions. +- Restructuring of project documentation. +- Build and test package documentation as a part of CI pipeline. +- Build and test package distribution as a part of CI pipeline. +- Check ``MANIFEST.in`` in a source package for completeness as a part of CI + pipeline. +- Added ``pytest`` and ``coverage[toml]`` to setuptools' ``extras_require``. `v0.5.0`_ - 30-August-2021 -------------------------- Added +++++ - - Support for Django 2.1 & 2.2 - - Added tox.ini targets - - Added secure redis backend URLs via ``rediss://`` - - Add ``cast=str`` to ``str()`` method +- Support for Django 2.1 & 2.2. +- Added tox.ini targets. +- Added secure redis backend URLs via ``rediss://``. +- Add ``cast=str`` to ``str()`` method. Fixed +++++ - - Fixed misspelling in the documentation +- Fixed misspelling in the documentation. Changed +++++++ - - Validate empty cache url and invalid cache schema - - Set ``long_description_content_type`` in setup - - Improved Django 1.11 database configuration support +- Validate empty cache url and invalid cache schema. +- Set ``long_description_content_type`` in setup. +- Improved Django 1.11 database configuration support. `v0.4.5`_ - 25-June-2018 ------------------------ Added +++++ - - Support for Django 2.0 - - Support for smart casting - - Support PostgreSQL unix domain socket paths - - Tip: Multiple env files +- Support for Django 2.0. +- Support for smart casting. +- Support PostgreSQL unix domain socket paths. +- Tip: Multiple env files. Changed +++++++ - - Fix parsing option values ``None``, ``True`` and ``False`` - - Order of importance of engine configuration in ``db_url_config`` +- Fix parsing option values ``None``, ``True`` and ``False``. +- Order of importance of engine configuration in ``db_url_config``. Removed +++++++ - - Remove ``django`` and ``six`` dependencies +- Remove ``django`` and ``six`` dependencies. `v0.4.4`_ - 21-August-2017 @@ -76,122 +108,125 @@ Removed Added +++++ - - Support for ``django-redis`` multiple locations (master/slave, shards) - - Support for Elasticsearch2 - - Support for Mysql-connector - - Support for ``pyodbc`` - - Add ``__contains__`` feature to Environ class +- Support for ``django-redis`` multiple locations (master/slave, shards). +- Support for Elasticsearch2. +- Support for Mysql-connector. +- Support for ``pyodbc``. +- Add ``__contains__`` feature to Environ class. Fixed +++++ - - Fix Path subtracting +- Fix Path subtracting. `v0.4.3`_ - 21-August-2017 -------------------------- Changed +++++++ - - Rollback the default Environ to ``os.environ`` +- Rollback the default Environ to ``os.environ``. `v0.4.2`_ - 13-April-2017 ------------------------- Added +++++ - - Confirm support for Django 1.11. - - Support for Redshift database URL +- Confirm support for Django 1.11. +- Support for Redshift database URL. Changed +++++++ - - Fix uwsgi settings reload problem (#55) - - Update support for ``django-redis`` urls (#109) +- Fix uwsgi settings reload problem + `#55 `_. +- Update support for ``django-redis`` urls + `#109 `_. `v0.4.1`_ - 13-November-2016 ---------------------------- Added +++++ - - Add support for Django 1.10 +- Add support for Django 1.10. Changed +++++++ - - Fix for unsafe characters into URLs - - Clarifying warning on missing or unreadable file. Thanks to @nickcatal - - Fix support for Oracle urls - - Fix support for ``django-redis`` +- Fix for unsafe characters into URLs. +- Clarifying warning on missing or unreadable file. + Thanks to `@nickcatal `_. +- Fix support for Oracle urls. +- Fix support for ``django-redis``. `v0.4`_ - 23-September-2015 --------------------------- Added +++++ - - New email schemes - ``smtp+ssl`` and ``smtp+tls`` (``smtps`` would be deprecated) - - Add tuple support. Thanks to @anonymouzz - - Add LDAP url support for database. Thanks to ``django-ldapdb`` +- New email schemes - ``smtp+ssl`` and ``smtp+tls`` (``smtps`` would be deprecated). +- Add tuple support. Thanks to `@anonymouzz `_. +- Add LDAP url support for database. Thanks to + `django-ldapdb/django-ldapdb `_. Changed +++++++ - - Fix non-ascii values (broken in Python 2.x) - - ``redis_cache`` replaced by ``django_redis`` - - Fix psql/pgsql url +- Fix non-ascii values (broken in Python 2.x). +- ``redis_cache`` replaced by ``django_redis``. +- Fix psql/pgsql url. `v0.3.1`_ - 19 Sep 2015 ----------------------- Added +++++ - - Added ``email`` as alias for ``email_url`` - - Django 1.7 is now supported - - Added LDAP scheme support for ``db_url_config`` +- Added ``email`` as alias for ``email_url``. +- Django 1.7 is now supported. +- Added LDAP scheme support for ``db_url_config``. Fixed +++++ - - Fixed typos in the documentation - - Fixed ``environ.Path.__add__`` to correctly handle plus operator - - Fixed ``environ.Path.__contains__`` to correctly work on Windows +- Fixed typos in the documentation. +- Fixed ``environ.Path.__add__`` to correctly handle plus operator. +- Fixed ``environ.Path.__contains__`` to correctly work on Windows. `v0.3`_ - 03-June-2014 ---------------------- Added +++++ - - Add cache url support - - Add email url support - - Add search url support +- Add cache url support. +- Add email url support. +- Add search url support. Changed +++++++ - - Rewriting README.rst +- Rewriting README.rst. -v0.2.1 19-April-2013 --------------------- +v0.2.1 - 19-April-2013 +---------------------- Changed +++++++ - - ``Env.__call__`` now uses ``Env.get_value`` instance method +- ``Env.__call__`` now uses ``Env.get_value`` instance method. -v0.2 16-April-2013 ------------------- +v0.2 - 16-April-2013 +-------------------- Added +++++ - - Add advanced float parsing (comma and dot symbols to separate thousands and decimals) +- Add advanced float parsing (comma and dot symbols to separate thousands and decimals). Fixed +++++ - - Fixed typos in the documentation +- Fixed typos in the documentation. -v0.1 2-April-2013 ------------------ +v0.1 - 2-April-2013 +------------------- Added +++++ - - Initial release +- Initial release. +.. _v0.7.0: https://github.com/joke2k/django-environ/compare/v0.6.0...v0.7.0 .. _v0.6.0: https://github.com/joke2k/django-environ/compare/v0.5.0...v0.6.0 .. _v0.5.0: https://github.com/joke2k/django-environ/compare/v0.4.5...v0.5.0 .. _v0.4.5: https://github.com/joke2k/django-environ/compare/v0.4.4...v0.4.5 .. _v0.4.4: https://github.com/joke2k/django-environ/compare/v0.4.3...v0.4.4 .. _v0.4.3: https://github.com/joke2k/django-environ/compare/v0.4.2...v0.4.3 .. _v0.4.2: https://github.com/joke2k/django-environ/compare/v0.4.1...v0.4.2 -.. _v0.4.1: https://github.com/joke2k/django-environ/compare/v0.4.0...v0.4.1 +.. _v0.4.1: https://github.com/joke2k/django-environ/compare/v0.4...v0.4.1 .. _v0.4: https://github.com/joke2k/django-environ/compare/v0.3.1...v0.4 .. _v0.3.1: https://github.com/joke2k/django-environ/compare/v0.3...v0.3.1 .. _v0.3: https://github.com/joke2k/django-environ/compare/v0.2.1...v0.3 -.. _`Keep a Changelog`: http://keepachangelog.com/en/1.0.0/ -.. _`Semantic Versioning`: http://semver.org/spec/v2.0.0.html diff --git a/README.rst b/README.rst index 4a9b8e98..df5a31ca 100644 --- a/README.rst +++ b/README.rst @@ -28,7 +28,7 @@ .. -teaser-begin- ``django-environ`` is the Python package that allows you to use -`Twelve-factor methodology `_ to configure your +`Twelve-factor methodology `_ to configure your Django application with environment variables. .. -teaser-end- @@ -92,13 +92,13 @@ environment variables obtained from an environment file and provided by the OS: The idea of this package is to unify a lot of packages that make the same stuff: Take a string from ``os.environ``, parse and cast it to some of useful python -typed variables. To do that and to use the `12factor `_ +typed variables. To do that and to use the `12factor `_ approach, some connection strings are expressed as url, so this package can parse it and return a ``urllib.parse.ParseResult``. These strings from ``os.environ`` are loaded from a ``.env`` file and filled in ``os.environ`` with ``setdefault`` method, to avoid to overwrite the real environ. -A similar approach is used in `Two Scoops of Django `_ -book and explained in `12factor-django `_ +A similar approach is used in `Two Scoops of Django `_ +book and explained in `12factor-django `_ article. @@ -120,7 +120,7 @@ Project Information =================== ``django-environ`` is released under the `MIT / X11 License `__, -its documentation lives at `Read the Docs `_, +its documentation lives at `Read the Docs `_, the code on `GitHub `_, and the latest release on `PyPI `_. diff --git a/docs/conf.py b/docs/conf.py index f11a4d50..17536146 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -56,7 +56,7 @@ def find_version(meta_file): "sphinx.ext.doctest", "sphinx.ext.intersphinx", "sphinx.ext.todo", - "notfound.extension" + "notfound.extension", ] # Add any paths that contain templates here, relative to this directory. @@ -66,7 +66,9 @@ def find_version(meta_file): source_suffix = ".rst" # Allow non-local URIs so we can have images in CHANGELOG etc. -suppress_warnings = ["image.nonlocal_uri"] +suppress_warnings = [ + "image.nonlocal_uri", +] # The master toctree document. master_doc = "index" diff --git a/docs/getting-started.rst b/docs/getting-started.rst index c1ae7a25..007e92f3 100644 --- a/docs/getting-started.rst +++ b/docs/getting-started.rst @@ -18,7 +18,7 @@ _________________________ ``django-environ`` is a Python-only package `hosted on PyPI `_. The recommended installation method is `pip `_-installing into a virtualenv: -.. code-block:: shell +.. code-block:: console $ python -m pip install django-environ @@ -34,7 +34,7 @@ So, you can also install the latest unreleased development version directly from ``develop`` branch on GitHub. It is a work-in-progress of a future stable release so the experience might be not as smooth.: -.. code-block:: shell +.. code-block:: console $ pip install -e git://github.com/joke2k/django-environ.git#egg=django-environ # OR @@ -114,7 +114,9 @@ FAQ ``django-environ`` will try to get and read ``.env`` file from the project root if you haven't specified the path for it when call ``read_env``. However, this is not the recommended way. When it is possible always specify - the path tho ``.env`` file. + the path tho ``.env`` file. Alternatively, you can use a trick with a + environment variable pointing to the actual location of .env file. + For details see ":ref:`multiple-env-files-label`". #. **What (where) is the root part of the project, is it part of the project where are settings?** diff --git a/docs/tips.rst b/docs/tips.rst index f82b0c4c..a09391f7 100644 --- a/docs/tips.rst +++ b/docs/tips.rst @@ -29,7 +29,7 @@ To disable it use ``env.smart_cast = False``. Multiple redis cache locations ============================== -For redis cache, `multiple master/slave or shard locations `_ can be configured as follows: +For redis cache, multiple master/slave or shard locations can be configured as follows: .. code-block:: shell @@ -119,17 +119,36 @@ Values that being with a ``$`` may be interpolated. Pass ``interpolate=True`` to FOO +.. _multiple-env-files-label: + Multiple env files ================== -It is possible to have multiple env files and select one using environment variables. +There is an ability point to the .env file location using an environment +variable. This feature may be convenient in a production systems with a +different .env file location. + +The following example demonstrates the above: + +.. code-block:: shell + + # /etc/environment file contents + DEBUG=False + +.. code-block:: shell + + # .env file contents + DEBUG=True .. code-block:: python env = environ.Env() env.read_env(env.str('ENV_PATH', '.env')) -Now ``ENV_PATH=other-env ./manage.py runserver`` uses ``other-env`` while ``./manage.py runserver`` uses ``.env``. + +Now ``ENV_PATH=/etc/environment ./manage.py runserver`` uses ``/etc/environment`` +while ``./manage.py runserver`` uses ``.env``. + Using Path objects when reading env =================================== diff --git a/docs/types.rst b/docs/types.rst index b2144d06..1e80e582 100644 --- a/docs/types.rst +++ b/docs/types.rst @@ -41,6 +41,9 @@ Supported types * ``search_url`` * Elasticsearch: ``elasticsearch://`` + * Elasticsearch2: ``elasticsearch2://`` + * Elasticsearch5: ``elasticsearch5://`` + * Elasticsearch7: ``elasticsearch7://`` * Solr: ``solr://`` * Whoosh: ``whoosh://`` * Xapian: ``xapian://`` diff --git a/environ/__init__.py b/environ/__init__.py index e307a317..bf8689a0 100644 --- a/environ/__init__.py +++ b/environ/__init__.py @@ -31,7 +31,7 @@ __copyright__ = 'Copyright (C) 2021 Daniele Faraglia' -__version__ = '0.6.0' +__version__ = '0.7.0' __license__ = 'MIT' __author__ = 'Daniele Faraglia' __author_email__ = 'daniele.faraglia@gmail.com' diff --git a/environ/environ.py b/environ/environ.py index 9561f190..0267add6 100644 --- a/environ/environ.py +++ b/environ/environ.py @@ -30,11 +30,11 @@ try: from os import PathLike + Openable = (str, PathLike) except ImportError: Openable = (str,) - logger = logging.getLogger(__name__) @@ -44,7 +44,7 @@ def _cast(value): # https://docs.python.org/3/library/ast.html#ast.literal_eval try: return ast.literal_eval(value) - except ValueError: + except (ValueError, SyntaxError): return value @@ -64,7 +64,6 @@ def __repr__(self): class Env: - """Provide scheme-based lookups of environment variables so that each caller doesn't have to pass in `cast` and `default` parameters. @@ -79,9 +78,11 @@ class Env: NOTSET = NoValue() BOOLEAN_TRUE_STRINGS = ('true', 'on', 'ok', 'y', 'yes', '1') URL_CLASS = ParseResult - DEFAULT_DATABASE_ENV = 'DATABASE_URL' POSTGRES_FAMILY = ['postgres', 'postgresql', 'psql', 'pgsql', 'postgis'] + ELASTICSEARCH_FAMILY = ['elasticsearch' + x for x in ['', '2', '5', '7']] + + DEFAULT_DATABASE_ENV = 'DATABASE_URL' DB_SCHEMES = { 'postgres': DJANGO_POSTGRES, 'postgresql': DJANGO_POSTGRES, @@ -146,6 +147,10 @@ class Env: "ElasticsearchSearchEngine", "elasticsearch2": "haystack.backends.elasticsearch2_backend." "Elasticsearch2SearchEngine", + "elasticsearch5": "haystack.backends.elasticsearch5_backend." + "Elasticsearch5SearchEngine", + "elasticsearch7": "haystack.backends.elasticsearch7_backend." + "Elasticsearch7SearchEngine", "solr": "haystack.backends.solr_backend.SolrEngine", "whoosh": "haystack.backends.whoosh_backend.WhooshEngine", "xapian": "haystack.backends.xapian_backend.XapianEngine", @@ -188,7 +193,10 @@ def bytes(self, var, default=NOTSET, encoding='utf8'): """ :rtype: bytes """ - return self.get_value(var, cast=str).encode(encoding) + value = self.get_value(var, cast=str, default=default) + if hasattr(value, 'encode'): + return value.encode(encoding) + return value def bool(self, var, default=NOTSET): """ @@ -262,6 +270,7 @@ def db_url(self, var=DEFAULT_DATABASE_ENV, default=NOTSET, engine=None): self.get_value(var, default=default), engine=engine ) + db = db_url def cache_url(self, var=DEFAULT_CACHE_ENV, default=NOTSET, backend=None): @@ -275,6 +284,7 @@ def cache_url(self, var=DEFAULT_CACHE_ENV, default=NOTSET, backend=None): self.url(var, default=default), backend=backend ) + cache = cache_url def email_url(self, var=DEFAULT_EMAIL_ENV, default=NOTSET, backend=None): @@ -288,6 +298,7 @@ def email_url(self, var=DEFAULT_EMAIL_ENV, default=NOTSET, backend=None): self.url(var, default=default), backend=backend ) + email = email_url def search_url(self, var=DEFAULT_SEARCH_ENV, default=NOTSET, engine=None): @@ -352,8 +363,9 @@ def get_value(self, var, cast=None, default=NOTSET, parse_default=False): value = default # Resolve any proxied values - if hasattr(value, 'startswith') and value.startswith('$'): - value = value.lstrip('$') + prefix = b'$' if isinstance(value, bytes) else '$' + if hasattr(value, 'startswith') and value.startswith(prefix): + value = value.lstrip(prefix) value = self.get_value(value, cast=cast, default=default) # Smart casting @@ -413,10 +425,10 @@ def parse_value(cls, value, cast): value = tuple([x for x in val if x]) elif cast is float: # clean string - float_str = re.sub(r'[^\d,\.]', '', value) + float_str = re.sub(r'[^\d,.-]', '', value) # split for avoid thousand separator and different # locale comma/dot symbol - parts = re.split(r'[,\.]', float_str) + parts = re.split(r'[,.]', float_str) if len(parts) == 1: float_str = parts[0] else: @@ -492,7 +504,7 @@ def db_url_config(cls, url, engine=None): if url.scheme == 'oracle': # Django oracle/base.py strips port and fails on non-string value if not config['PORT']: - del(config['PORT']) + del (config['PORT']) else: config['PORT'] = str(config['PORT']) @@ -646,7 +658,7 @@ def search_url_config(cls, url, engine=None): config["ENGINE"] = cls.SEARCH_SCHEMES[url.scheme] # check commons params - params = {} + params = {} # type: dict if url.query: params = parse_qs(url.query) if 'EXCLUDED_INDEXES' in params.keys(): @@ -665,7 +677,7 @@ def search_url_config(cls, url, engine=None): if url.scheme == 'simple': return config - elif url.scheme in ['solr', 'elasticsearch', 'elasticsearch2']: + elif url.scheme in ['solr'] + cls.ELASTICSEARCH_FAMILY: if 'KWARGS' in params.keys(): config['KWARGS'] = params['KWARGS'][0] @@ -681,8 +693,7 @@ def search_url_config(cls, url, engine=None): config['TIMEOUT'] = cls.parse_value(params['TIMEOUT'][0], int) return config - if url.scheme in ['elasticsearch', 'elasticsearch2']: - + if url.scheme in cls.ELASTICSEARCH_FAMILY: split = path.rsplit("/", 1) if len(split) > 1: @@ -728,7 +739,7 @@ def read_env(cls, env_file=None, **overrides): called read_env. Refs: - - http://www.wellfireinteractive.com/blog/easier-12-factor-django/ + - https://wellfire.co/learn/easier-12-factor-django - https://gist.github.com/bennylope/2999704 """ if env_file is None: @@ -738,7 +749,7 @@ def read_env(cls, env_file=None, **overrides): '.env' ) if not os.path.exists(env_file): - warnings.warn( + logger.info( "%s doesn't exist - if you're not configuring your " "environment separately, create one." % env_file) return @@ -751,8 +762,8 @@ def read_env(cls, env_file=None, **overrides): with env_file as f: content = f.read() except OSError: - warnings.warn( - "Error reading %s - if you're not configuring your " + logger.info( + "%s not found - if you're not configuring your " "environment separately, check this." % env_file) return @@ -776,7 +787,6 @@ def read_env(cls, env_file=None, **overrides): class Path: - """Inspired to Django Two-scoops, handling File Paths in Settings.""" def path(self, *paths, **kwargs): diff --git a/setup.py b/setup.py index 34891de3..24e8a85f 100644 --- a/setup.py +++ b/setup.py @@ -187,7 +187,9 @@ def get_version_string(): # Project's URLs PROJECT_URLS = { - 'Documentation': 'https://django-environ.readthedocs.io', + 'Documentation': find_meta('url'), + 'Funding': 'https://opencollective.com/django-environ', + 'Say Thanks!': 'https://saythanks.io/to/joke2k', 'Changelog': '{}/en/latest/changelog.html'.format(find_meta('url')), 'Bug Tracker': 'https://github.com/joke2k/django-environ/issues', 'Source Code': 'https://github.com/joke2k/django-environ', diff --git a/tests/conftest.py b/tests/conftest.py index 32a19cdf..38550f73 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -19,12 +19,6 @@ def solr_url(): return 'solr://127.0.0.1:8983/solr' -@pytest.fixture -def elasticsearch_url(): - """Return Elasticsearch URL.""" - return 'elasticsearch://127.0.0.1:9200/index' - - @pytest.fixture def whoosh_url(): """Return Whoosh URL.""" diff --git a/tests/fixtures.py b/tests/fixtures.py index a476e298..f685f5cb 100644 --- a/tests/fixtures.py +++ b/tests/fixtures.py @@ -28,7 +28,7 @@ class FakeEnv: EXPORTED = 'exported var' @classmethod - def generateData(cls): + def generate_data(cls): return dict(STR_VAR='bar', MULTILINE_STR_VAR='foo\\nbar', INT_VAR='42', @@ -36,10 +36,20 @@ def generateData(cls): FLOAT_COMMA_VAR='33,3', FLOAT_STRANGE_VAR1='123,420,333.3', FLOAT_STRANGE_VAR2='123.420.333,3', - BOOL_TRUE_VAR='1', - BOOL_TRUE_VAR2='True', - BOOL_FALSE_VAR='0', - BOOL_FALSE_VAR2='False', + FLOAT_NEGATIVE_VAR='-1.0', + BOOL_TRUE_STRING_LIKE_INT='1', + BOOL_TRUE_INT=1, + BOOL_TRUE_STRING_LIKE_BOOL='True', + BOOL_TRUE_STRING_1='on', + BOOL_TRUE_STRING_2='ok', + BOOL_TRUE_STRING_3='yes', + BOOL_TRUE_STRING_4='y', + BOOL_TRUE_STRING_5='true', + BOOL_TRUE_BOOL=True, + BOOL_FALSE_STRING_LIKE_INT='0', + BOOL_FALSE_INT=0, + BOOL_FALSE_STRING_LIKE_BOOL='False', + BOOL_FALSE_BOOL=False, PROXIED_VAR='$STR_VAR', INT_LIST='42,33', INT_TUPLE='(42,33)', diff --git a/tests/test_cache.py b/tests/test_cache.py index 13077a53..42c72bbc 100644 --- a/tests/test_cache.py +++ b/tests/test_cache.py @@ -149,3 +149,80 @@ def test_unknown_backend(): def test_empty_url_is_mapped_to_empty_config(): assert Env.cache_url_config('') == {} assert Env.cache_url_config(None) == {} + + +@pytest.mark.parametrize( + 'chars', + ['!', '$', '&', "'", '(', ')', '*', '+', ';', '=', '-', '.', '-v1.2'] +) +def test_cache_url_password_using_sub_delims(monkeypatch, chars): + """Ensure CACHE_URL passwords may contains some unsafe characters. + + See: https://github.com/joke2k/django-environ/issues/200 for details.""" + url = 'rediss://enigma:secret{}@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + result = env.cache_url_config(url) + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + url = 'rediss://enigma:sec{}ret@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + result = env.cache_url_config(url) + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + url = 'rediss://enigma:{}secret@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + result = env.cache_url_config(url) + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + +@pytest.mark.parametrize( + 'chars', ['%3A', '%2F', '%3F', '%23', '%5B', '%5D', '%40', '%2C'] +) +def test_cache_url_password_using_gen_delims(monkeypatch, chars): + """Ensure CACHE_URL passwords may contains %-encoded characters. + + See: https://github.com/joke2k/django-environ/issues/200 for details.""" + url = 'rediss://enigma:secret{}@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + url = 'rediss://enigma:sec{}ret@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url + + url = 'rediss://enigma:{}secret@ondigitalocean.com:25061/2'.format(chars) + monkeypatch.setenv('CACHE_URL', url) + env = Env() + + result = env.cache() + assert result['BACKEND'] == 'django_redis.cache.RedisCache' + assert result['LOCATION'] == url diff --git a/tests/test_env.py b/tests/test_env.py index c4807de0..efb09fbc 100644 --- a/tests/test_env.py +++ b/tests/test_env.py @@ -26,7 +26,7 @@ def setup_method(self, method): class. setup_method is invoked for every test method of a class. """ self.old_environ = os.environ - os.environ = Env.ENVIRON = FakeEnv.generateData() + os.environ = Env.ENVIRON = FakeEnv.generate_data() self.env = Env() def teardown_method(self, method): @@ -65,8 +65,16 @@ def test_str(self, var, val, multiline): assert self.env(var) == val assert self.env.str(var, multiline=multiline) == val - def test_bytes(self): - assert_type_and_value(bytes, b'bar', self.env.bytes('STR_VAR')) + @pytest.mark.parametrize( + 'var,val,default', + [ + ('STR_VAR', b'bar', Env.NOTSET), + ('NON_EXISTENT_BYTES_VAR', b'some-default', b'some-default'), + ('NON_EXISTENT_STR_VAR', b'some-default', 'some-default'), + ] + ) + def test_bytes(self, var, val, default): + assert_type_and_value(bytes, val, self.env.bytes(var, default=default)) def test_int(self): assert_type_and_value(int, 42, self.env('INT_VAR', cast=int)) @@ -75,23 +83,41 @@ def test_int(self): def test_int_with_none_default(self): assert self.env('NOT_PRESENT_VAR', cast=int, default=None) is None - def test_float(self): - assert_type_and_value(float, 33.3, self.env('FLOAT_VAR', cast=float)) - assert_type_and_value(float, 33.3, self.env.float('FLOAT_VAR')) - - assert_type_and_value(float, 33.3, self.env('FLOAT_COMMA_VAR', cast=float)) - assert_type_and_value(float, 123420333.3, self.env('FLOAT_STRANGE_VAR1', cast=float)) - assert_type_and_value(float, 123420333.3, self.env('FLOAT_STRANGE_VAR2', cast=float)) - - def test_bool_true(self): - assert_type_and_value(bool, True, self.env('BOOL_TRUE_VAR', cast=bool)) - assert_type_and_value(bool, True, self.env('BOOL_TRUE_VAR2', cast=bool)) - assert_type_and_value(bool, True, self.env.bool('BOOL_TRUE_VAR')) + @pytest.mark.parametrize( + 'value,variable', + [ + (33.3, 'FLOAT_VAR'), + (33.3, 'FLOAT_COMMA_VAR'), + (123420333.3, 'FLOAT_STRANGE_VAR1'), + (123420333.3, 'FLOAT_STRANGE_VAR2'), + (-1.0, 'FLOAT_NEGATIVE_VAR'), + ] + ) + def test_float(self, value, variable): + assert_type_and_value(float, value, self.env.float(variable)) + assert_type_and_value(float, value, self.env(variable, cast=float)) - def test_bool_false(self): - assert_type_and_value(bool, False, self.env('BOOL_FALSE_VAR', cast=bool)) - assert_type_and_value(bool, False, self.env('BOOL_FALSE_VAR2', cast=bool)) - assert_type_and_value(bool, False, self.env.bool('BOOL_FALSE_VAR')) + @pytest.mark.parametrize( + 'value,variable', + [ + (True, 'BOOL_TRUE_STRING_LIKE_INT'), + (True, 'BOOL_TRUE_STRING_LIKE_BOOL'), + (True, 'BOOL_TRUE_INT'), + (True, 'BOOL_TRUE_BOOL'), + (True, 'BOOL_TRUE_STRING_1'), + (True, 'BOOL_TRUE_STRING_2'), + (True, 'BOOL_TRUE_STRING_3'), + (True, 'BOOL_TRUE_STRING_4'), + (True, 'BOOL_TRUE_STRING_5'), + (False, 'BOOL_FALSE_STRING_LIKE_INT'), + (False, 'BOOL_FALSE_INT'), + (False, 'BOOL_FALSE_STRING_LIKE_BOOL'), + (False, 'BOOL_FALSE_BOOL'), + ] + ) + def test_bool_true(self, value, variable): + assert_type_and_value(bool, value, self.env.bool(variable)) + assert_type_and_value(bool, value, self.env(variable, cast=bool)) def test_proxied_value(self): assert self.env('PROXIED_VAR') == 'bar' @@ -250,8 +276,10 @@ def test_path(self): def test_smart_cast(self): assert self.env.get_value('STR_VAR', default='string') == 'bar' - assert self.env.get_value('BOOL_TRUE_VAR', default=True) - assert self.env.get_value('BOOL_FALSE_VAR', default=True) is False + assert self.env.get_value('BOOL_TRUE_STRING_LIKE_INT', default=True) + assert not self.env.get_value( + 'BOOL_FALSE_STRING_LIKE_INT', + default=True) assert self.env.get_value('INT_VAR', default=1) == 42 assert self.env.get_value('FLOAT_VAR', default=1.2) == 33.3 @@ -314,7 +342,7 @@ def setup_method(self, method): """ super().setup_method(method) - self.CONFIG = FakeEnv.generateData() + self.CONFIG = FakeEnv.generate_data() class MyEnv(Env): ENVIRON = self.CONFIG diff --git a/tests/test_env.txt b/tests/test_env.txt index f00ce490..dfbd61ef 100644 --- a/tests/test_env.txt +++ b/tests/test_env.txt @@ -1,5 +1,4 @@ DICT_VAR=foo=bar,test=on -BOOL_FALSE_VAR2=False DATABASE_MYSQL_URL=mysql://bea6eb0:69772142@us-cdbr-east.cleardb.com/heroku_97681?reconnect=true DATABASE_MYSQL_GIS_URL=mysqlgis://user:password@127.0.0.1/some_database CACHE_URL=memcache://127.0.0.1:11211 @@ -7,16 +6,27 @@ CACHE_REDIS=rediscache://127.0.0.1:6379/1?client_class=django_redis.client.Defau EMAIL_URL=smtps://user@domain.com:password@smtp.example.com:587 URL_VAR=http://www.google.com/ PATH_VAR=/home/dev -BOOL_FALSE_VAR=0 -BOOL_TRUE_VAR2=True +BOOL_TRUE_STRING_LIKE_INT='1' +BOOL_TRUE_INT=1 +BOOL_TRUE_STRING_LIKE_BOOL='True' +BOOL_TRUE_STRING_1='on' +BOOL_TRUE_STRING_2='ok' +BOOL_TRUE_STRING_3='yes' +BOOL_TRUE_STRING_4='y' +BOOL_TRUE_STRING_5='true' +BOOL_TRUE_BOOL=True +BOOL_FALSE_STRING_LIKE_INT='0' +BOOL_FALSE_INT=0 +BOOL_FALSE_STRING_LIKE_BOOL='False' +BOOL_FALSE_BOOL=False DATABASE_SQLITE_URL=sqlite:////full/path/to/your/database/file.sqlite JSON_VAR={"three": 33.44, "two": 2, "one": "bar"} -BOOL_TRUE_VAR=1 DATABASE_URL=postgres://uf07k1:wegauwhg@ec2-107-21-253-135.compute-1.amazonaws.com:5431/d8r82722 FLOAT_VAR=33.3 FLOAT_COMMA_VAR=33,3 FLOAT_STRANGE_VAR1=123,420,333.3 FLOAT_STRANGE_VAR2=123.420.333,3 +FLOAT_NEGATIVE_VAR=-1.0 PROXIED_VAR=$STR_VAR EMPTY_LIST= INT_VAR=42 diff --git a/tests/test_schema.py b/tests/test_schema.py index ba296823..7a7f62ec 100644 --- a/tests/test_schema.py +++ b/tests/test_schema.py @@ -19,7 +19,7 @@ def setup_module(): global _old_environ _old_environ = os.environ - os.environ = Env.ENVIRON = FakeEnv.generateData() + os.environ = Env.ENVIRON = FakeEnv.generate_data() def teardown_module(): diff --git a/tests/test_search.py b/tests/test_search.py index 99666d59..a81471e5 100644 --- a/tests/test_search.py +++ b/tests/test_search.py @@ -32,14 +32,26 @@ def test_solr_multicore_parsing(solr_url): assert 'PATH' not in url -def test_elasticsearch_parsing(elasticsearch_url): +@pytest.mark.parametrize( + 'url,engine', + [ + ('elasticsearch://127.0.0.1:9200/index', + 'elasticsearch_backend.ElasticsearchSearchEngine'), + ('elasticsearch2://127.0.0.1:9200/index', + 'elasticsearch2_backend.Elasticsearch2SearchEngine'), + ('elasticsearch5://127.0.0.1:9200/index', + 'elasticsearch5_backend.Elasticsearch5SearchEngine'), + ('elasticsearch7://127.0.0.1:9200/index', + 'elasticsearch7_backend.Elasticsearch7SearchEngine'), + ] +) +def test_elasticsearch_parsing(url, engine): + """Ensure all supported Elasticsearch engines are recognized.""" timeout = 360 - url = '{}?TIMEOUT={}'.format(elasticsearch_url, timeout) + url = '{}?TIMEOUT={}'.format(url, timeout) url = Env.search_url_config(url) - assert url['ENGINE'] == ( - 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine' - ) + assert url['ENGINE'] == 'haystack.backends.{}'.format(engine) assert 'INDEX_NAME' in url.keys() assert url['INDEX_NAME'] == 'index' assert 'TIMEOUT' in url.keys() diff --git a/tests/test_utils.py b/tests/test_utils.py new file mode 100644 index 00000000..523a72d3 --- /dev/null +++ b/tests/test_utils.py @@ -0,0 +1,22 @@ +# This file is part of the django-environ. +# +# Copyright (c) 2021, Serghei Iakovlev +# Copyright (c) 2013-2021, Daniele Faraglia +# +# For the full copyright and license information, please view +# the LICENSE.txt file that was distributed with this source code. + +import pytest +from environ.environ import _cast + + +@pytest.mark.parametrize( + 'literal', + ['anything-', 'anything*', '*anything', 'anything.', + 'anything.1', '(anything', 'anything-v1.2', 'anything-1.2', 'anything='] +) +def test_cast(literal): + """Safely evaluate a string containing an invalid Python literal. + + See https://github.com/joke2k/django-environ/issues/200 for details.""" + assert _cast(literal) == literal diff --git a/tox.ini b/tox.ini index a32a27ea..4877c140 100644 --- a/tox.ini +++ b/tox.ini @@ -14,6 +14,7 @@ minversion = 3.22 envlist = build coverage-report + linkcheck docs lint manifest @@ -69,6 +70,24 @@ commands_pre = python -m pip install . commands = flake8 environ setup.py +[testenv:linkcheck] +description = Check external links in the package documentation +# Keep basepython in sync with .readthedocs.yml and docs.yml +# (GitHub Action Workflow). +basepython = python3.8 +extras = docs +commands = + {envpython} -m sphinx \ + -j auto \ + -b linkcheck \ + {tty:--color} \ + -n -W -a \ + --keep-going \ + -d {envtmpdir}/doctrees \ + docs \ + docs/_build/linkcheck +isolated_build = true + [testenv:docs] description = Build package documentation (HTML) # Keep basepython in sync with .readthedocs.yml and docs.yml @@ -76,9 +95,30 @@ description = Build package documentation (HTML) basepython = python3.8 extras = docs commands = - sphinx-build -n -T -W -b html -d {envtmpdir}/doctrees docs docs/_build/html - sphinx-build -n -T -W -b doctest -d {envtmpdir}/doctrees docs docs/_build/html - python -m doctest AUTHORS.rst CHANGELOG.rst CONTRIBUTING.rst README.rst SECURITY.rst + {envpython} -m sphinx \ + -j auto \ + -b html \ + {tty:--color} \ + -n -T -W \ + -d {envtmpdir}/doctrees \ + docs \ + docs/_build/html + + {envpython} -m sphinx \ + -j auto \ + -b doctest \ + {tty:--color} \ + -n -T -W \ + -d {envtmpdir}/doctrees \ + docs \ + docs/_build/doctest + + {envpython} -m doctest \ + AUTHORS.rst \ + CHANGELOG.rst \ + CONTRIBUTING.rst \ + README.rst \ + SECURITY.rst [testenv:manifest] description = Check MANIFEST.in in a source package for completeness