Option to regard invalid configuration as an error

[ollik1]

Current problem

Currently, if unrecognized options are used, pylint prints a warning message but continues and eventually returns 0. This can be problematic especially when running pylint in CICD pipelines and pylint updates in registry.

.pylintrc:1: [E0015(unrecognized-option), ] Unrecognized option found: optimize-ast, files-output, no-space-check, function-name-hint, variable-name-hint, const-name-hint, attr-name-hint, argument-name-hint, class-attribute-name-hint, inlinevar-name-hint, class-name-hint, module-name-hint, method-name-hint

--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

Desired solution

Return non-zero value when unrecognized options are found or provide an option for enabling this.

Additional context

No response

[Pierre-Sassoulas]

pylint should not return with exit 0 if it emits an error level message, unless other option were provided. If that's the case then it's a bug and we need to fix it.

What about using --fail-on=unrecognized-option ?

[ollik1]

pylint should not return with exit 0 if it emits an error level message, unless other option were provided. If that's the case then it's a bug and we need to fix it.

What about using --fail-on=unrecognized-option ?

This option doesn't change the behavior. Here are quick steps to re-produce the issue using Python 3.8 (I'm using # to denote stdout):

mkdir unrecognized-option-test
cd unrecognized-option-test
python -m venv venv
pip install pylint==2.14.4
pylint --version
# pylint 2.14.4
# astroid 2.11.7
# Python 3.8.12 (default, Feb 25 2022, 21:43:33)
# [Clang 13.0.0 (clang-1300.0.27.3)]

cat <<EOF >> .pylintrc
[MESSAGES CONTROL]
disable=missing-module-docstring
[FORMAT]
no-space-check=trailing-comma,dict-separator
EOF

mkdir src
cat <<EOF >> src/test.py
print("hello")
EOF

pylint --fail-on=unrecognized-option src
# ************* Module /tmp/unrecognized-option-test/.pylintrc
# .pylintrc:1:0: E0015: Unrecognized option found: no-space-check (unrecognized-option)
#
# --------------------------------------------------------------------
# Your code has been rated at 10.00/10 (previous run: 0.00/10, +10.00)

echo $?
# 0

[ollik1]

This seems to be the problematic code path. When the execution reaches this point, self.linter.msg_status is 2 but 0 is returned if the score is high enough (and the configuration errors don't affect the score). I wonder if it would be better to exit with 32 after parsing configuration file if any configuration errors are found. This would make the behavior consistent with configuration using program arguments.

[DanielNoord]

I think some people depend on incorrect configurations being allowed, so I don't think we should always exit with an error.

Perhaps we should look into letting the configuration error affect the score? That way those who want to have incorrect configurations can just set their fail-under to a lower limit. @Pierre-Sassoulas Do you know why configuration errors don't affect the score?

[Pierre-Sassoulas]

Do you know why configuration errors don't affect the score?

(Maybe because it was designed as a wart on top of the existing code ? If that's the case making it generic it would be great).

some people depend on incorrect configurations being allowed

Might be this, but frankly it's possible to disable, it's possible to have multiple configuration file, at some point we need to say no to some features and I'd break thing that do not make sense or are too hard to maintain in 3.0.

[jacobtylerwalls]

Interestingly this works as expected for CLI args, just not for config file options.

[jacobtylerwalls]

We should just throw the same way #7931 does (do it in every code path)