Add --strict option to fail on informational messages

[xavfernandez]

Is your feature request related to a problem? Please describe

A bunch of useful informational messages exist in pylint but enforcing them in CI is tricky since their presence doesn't affect the exit code.

Describe the solution you'd like

The activation of this --strict (or --fail-on-informational) option would simply change
MSG_TYPES_STATUS['I'] value to a none-zero value to be determined (I'd go with 1)

https://github.com/PyCQA/pylint/blob/f2f4e6f42416644471ab003d4df7ecf052c3c411/pylint/constants.py#L33

to make pylint fail when an informational message is produced.

Additional context

This could be an alternative to #3251

[xavfernandez]

In the meantime, a fake plugin pylint-strict-informational that does exactly that has been published.

To use, pip install pylint-strict-informational then use pylint --load-plugins=pylint_strict_informational (or set it in your configuration file).
(From an original idea of @dbaty).

[PCManticore]

A strict mode could make sense for other checks as well. It's definitely something we should consider.

[papb]

@xavfernandez Unfortunately pylint-strict-informational did not work for me (Polyconseil/pylint-strict-informational#2).

I ended up with the following workaround (executed before calling pylint):

find .venv/lib -path '*/pylint/constants.py' -exec sed -r -i -e 's@(^MSG_TYPES_STATUS = .*):\s*0,(.*)@\1: 4,\2  # monkey-patched, waiting for pylint#3363@g' {} ';'

[MarkCBell]

Note, since ab094ae, it is not just enough to change the MSG_TYPES_STATUS of the informational type messages. This is because this commit means that a shortcut exit(0) is applied whenever a run completes with a score >= config.fail_under (which is 10.0 by default). Since informational messages do not affect the score, if only informational messages are raised during a run then the run completes with a perfect score and so this shortcut is applied and the job exits with return code 0 regardless of the value of self.linter.msg_status.

This is the reason that pylint-strict-informational no longer works (unless you use an option like --fail-under=11 to prevent this shortcut from ever being applied).

[papb]

Hello @Pierre-Sassoulas and everyone, sorry to take long, but: why was this issue closed? PR #4713 does not seem to add a flag to allow failing on any informational message.

At least it seems I can now achieve it by manually adding --fail-on for all existing informational messages, but I don't like that. Thoughts?

[papb]

Never mind, adding the following to my .pylintrc worked:

evaluation=10.0 - ((float(5 * error + warning + refactor + convention + info) / statement) * 10)