Using --fail-on does not always result in a failing return code when that issue is present

[MarkCBell]

Steps to reproduce

Run: pylint --fail-on=useless-suppression a.py

When a.py contains:

"""Test file to trigger an useless-suppression error msg"""
# pylint: disable=broad-except
pass

Current behavior

Result of pylint --fail-on=useless-suppression a.py:

************* Module useless_suppression
tests/examples/useless_suppression.py:2:0: I0021: Useless suppression of 'broad-except' (useless-suppression)

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

However this process finishes with exit code 0:

> echo $?
0

Expected behavior

pylint should have a non-zero return code since a failing issue was matched. However when failing matches occur the behavor is to exit with self.linter.msg_status, unfortunately this is zero if only "info" type issues have occurred.

pylint --version output

Result of pylint --version output:

pylint 2.9.4-dev0
astroid 2.6.2
Python 3.8.10 (default, Jun  2 2021, 10:49:15) 
[GCC 9.4.0]

[Pierre-Sassoulas]

Duplicate of #4296 that you opened yourself and #3363 in which you participated ? If you're interested in the issue you can open a merge request, but opening more issues is not going to make the problem fix itself.

[MarkCBell]

So I'm not certain that this is an exact duplicate since this is specifically about the --fail-on option, although the root cause is certainly very similar. I've submit my merge request #4713 referencing these as you suggested.