Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved Django settings handling #287

Merged
merged 8 commits into from Aug 28, 2020

Conversation

carlio
Copy link
Collaborator

@carlio carlio commented Aug 26, 2020

This revisits #277 and #243 @atodorov @alejandro-angulo

I had a new attempt atdealing with django settings required by the foreign key string handling and came up with this solution:

  1. as currently, if DJANGO_SETTINGS_MODULE is specified, those settings will be used.
  2. a commandline option can be used instead - for example pylint --load-plugins=pylint_django --django-settings-module=myproject.settings which I assume would work from a .pylintrc too.
  3. if neither of these are set, then Django is configured using the default settings. In this case, an error message is emitted (E5110) instructing the user to configure Django manually to improve linting. I set this as ERROR because it's more severe than just a warning since it is kind of an error in the testing (not in the code being tested, but the testing CI setup). It is not fatal, however, as it can continue to run.

This is done by moving the Foreign Key checks into a checker rather than loading the transforms in when the plugin is loaded. This allows for using the open() "setup" method to do any pre-check startup, in this case, running the Django settings.configure() method manually rather than allowing Django to do it by itself if it has an environment variable.

Using a checker also means that a commandline option can be used as an additional option for configuring Django settings, which will allow it to be used in pre-commit for example (thus fixing #286)

Note: this also had to remove the --no-binary :all: from the scripts/build.sh sanity check because it seems that a newer release of isort (which pylint depends on) requires clikit and when --no-binary is used, pylint-django fails to install. I'm not sure of the consequences of this but I assume that if pylint needs it to install, we have to accept that. This is not related to the main contents of the PR, just something I had to do to get the latest versions to pass CI.

…he `open()` pre-configuration. This allows the django settings module to be set from a command-line option or .pylintrc option as well as an environment variable (refs #286). This commit also uses default Django settings if none are specified, raising an error to the user to inform them that it is better to set it explicitly, but avoiding the need to raise a Fatal or RuntimeError if Django is not configured (refs #277 and #243)
…king - alternative is to manually supply a basic django settings file which seems like it would clog up the repo more than one single disable flag in the tox file
…nary dependancy on clikit, so either we have to pin isort to an earlier version or prevent checking that pylint-django can be installed with --no-binary (since, presumably, isort and therefore pylint now require binary package dependencies)
@coveralls
Copy link

coveralls commented Aug 26, 2020

Pull Request Test Coverage Report for Build 1233

  • 30 of 50 (60.0%) changed or added relevant lines in 7 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-1.6%) to 86.988%

Changes Missing Coverage Covered Lines Changed/Added Lines %
pylint_django/transforms/foreignkey.py 0 1 0.0%
pylint_django/checkers/django_installed.py 0 2 0.0%
pylint_django/checkers/foreign_key_strings.py 25 42 59.52%
Totals Coverage Status
Change from base Build 1214: -1.6%
Covered Lines: 722
Relevant Lines: 830

💛 - Coveralls

@carlio carlio force-pushed the feature/improve-django-settings-handling branch from a5ff57b to cc77809 Compare August 26, 2020 11:35
@carlio carlio force-pushed the feature/improve-django-settings-handling branch from cc77809 to f7b556b Compare August 26, 2020 11:59
Copy link
Contributor

@alejandro-angulo alejandro-angulo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for cleaning up foreign key string issues 👍 . PR lgtm

Should issues be created for the new TODOs? Not sure how these things are usually handled in this repo but I've found that TODOs tend to not get addressed without a related ticket/issue.

Copy link
Contributor

@atodorov atodorov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me, just a couple of nit picks and comments.

pylint_django/checkers/django_installed.py Outdated Show resolved Hide resolved
pylint_django/checkers/foreign_key_strings.py Outdated Show resolved Hide resolved
@@ -156,7 +156,7 @@ def func(apps, schema_editor):
return is_migrations_module(node.parent)


def load_configuration(linter):
def load_configuration(linter): # TODO this is redundant and can be removed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for adding the TODOs as separate issues so we don't forget about them. Interestingly enough the pylint test job in Travis should be catching this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - I had initially not bothered as I had planned to immediately address them in a new PR but I got distracted / didn't get time. Therefore I've made #289 and #288



foreignkey.add_transform(astroid.MANAGER)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could benefit from a comment either here or on the doc-string text and/or the foreignkey module to explicitly state why foreignkey.add_transform() isn't called like for the other modules.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a module-level docstring explaining this already which I think is sufficient: https://github.com/PyCQA/pylint-django/blob/feature/improve-django-settings-handling/pylint_django/transforms/__init__.py#L1

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also added this issue as a discussion for moving all transforms/augmentations into a checker rather than being loaded immediately by the plugin: #290

pylint_django/transforms/foreignkey.py Show resolved Hide resolved
@@ -69,7 +69,7 @@ echo "..... Trying to install the new tarball inside a virtualenv"
# note: installs with the optional dependency to verify this is still working
virtualenv .venv/test-tarball
source .venv/test-tarball/bin/activate
pip install --no-binary :all: -f dist/ pylint_django[with_django]
pip install -f dist/ pylint_django[with_django]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for this line & the reason why it is failing is the same - it tries installing pylint-django and all of its dependencies from tartballs. One of them is either new or stopped shipping tarballs, and only has wheel packages. See 10 lines below for how we deal with another package which doesn't ship wheels.

My preferred way for dealing with this is to find which package is the one not providing tarballs and

  1. Open an issue against it
  2. Install it explicitly by itself.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I'll accept this in our own CI while I track down which dependency is the guilty party - I am fairly sure it's clikit and I'll raise a PR/issue there if that's true or as you suggest, explicitly install it ourselves.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a new issue to separate this out: #291

@carlio carlio merged commit 7721691 into master Aug 28, 2020
@carlio carlio deleted the feature/improve-django-settings-handling branch August 28, 2020 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants