Invoking Pylint programmatically
Question:
I’d like to invoke the Pylint checker, limited to the error signalling part, as part of my unit testing. So I checked the Pylint executable script, got to the pylint.lint.Run helper class and there I got lost in a quite long __init__ function, ending with a call to sys.exit().
Anybody ever tried and managed to do so?
The dream-plan would be this:
if __name__ == '__main__':
import pylint.lint
pylint.lint.something(__file__, justerrors=True)
# now continue with unit testing
Any hints? Other than "copy the __init__ method and skip the sys.exit()", I mean?
I don’t need the tests to be run by Pylint, it might as well be pyflakes or other software: feel free to suggest alternatives.
Answers:
Take a look at the pylint/epylint.py file which contains two different ways to start Pylint programmatically.
You can also simply call
from pylint.lint import Run
Run(['--errors-only', 'myfile.py'])
for instance.
I’m glad I came across this. I used some of the answers here and some initiative to come up with:
# a simple class with a write method
class WritableObject:
def __init__(self):
self.content = []
def write(self, string):
self.content.append(string)
pylint_output = WritableObject()
pylint = lint.Run(args, reporter=ParseableTextReporter(pylint_output), exit=False)
Args in the above is a list of strings eg. [“-r”, “n”, “myfile.py”]
I got the same problem recently.
syt is right, pylint.epylint got several methods in there. However they all call a subprocess in which python is launched again. In my case, this was getting quite slow.
Building from mcarans answer, and finding that there is a flag exit, I did the following
class WritableObject(object):
"dummy output stream for pylint"
def __init__(self):
self.content = []
def write(self, st):
"dummy write"
self.content.append(st)
def read(self):
"dummy read"
return self.content
def run_pylint(filename):
"run pylint on the given file"
from pylint import lint
from pylint.reporters.text import TextReporter
ARGS = ["-r","n", "--rcfile=rcpylint"] # put your own here
pylint_output = WritableObject()
lint.Run([filename]+ARGS, reporter=TextReporter(pylint_output), exit=False)
for l in pylint_output.read():
do what ever you want with l...
which is about 3 times faster in my case.
With this I have been going through a whole project, using full output to check each source file, point errors, and rank all files from their note.
Instead of creating a WritableObject class we can use StringIO. StringIO contains write method.
import sys
try:
from io import StringIO
except:
from StringIO import StringIO
stdout = sys.stdout
sys.stdout = StringIO()
ARGS = ["-r","n", "--rcfile=rcpylint"]
r = lint.Run(['../test.py']+ARGS, exit=False)
test = sys.stdout.getvalue()
sys.stdout.close()
sys.stdout = stdout
print (test.split('n'))
Source:
Another entry point for pylint is the epylint.py_run function, that implement the stdout and stderr interception. However, as shown in the following code, pylint seems to not write its reports in stdout:
from pylint import epylint
pylint_stdout, pylint_stderr = epylint.py_run(__file__, return_std=True)
print(pylint_stdout.getvalue()) # -> there is just the final rank, no report nor message
print(pylint_stderr.getvalue())
Now, i found that pylint from API and pylint from CLI do not use the same default parameters. So, you just have to provides the parameters you need to pylint.
from pylint import epylint
options = '--enable=all' # all messages will be shown
options += '--reports=y' # also print the reports (ascii tables at the end)
pylint_stdout, pylint_stderr = epylint.py_run(__file__ + ' ' + options, return_std=True)
print(pylint_stdout.getvalue())
print(pylint_stderr.getvalue())
As described here, pylint will perform the parsing itself, and will correctly output the expected results in stdout.
NOTE:
At some point pylint changed the interface. The above examples need to replace exit=False with do_exit=False. (@mad7777, @amit-tripathi)
(Learned as per https://github.com/carsongee/pytest-pylint/issues/80)
Here is a wrapper I use to programmatically call pylint so I have a –fail-under arg to overwite the default pylint exit code (usefull for CI). This snippet was tested using pylint 2.3.1
""" Execute pylint and fail if score not reached. """
import argparse
import sys
from pylint import lint
desc = "PyLint wrapper that add the --fail-under option."
" All other arguments are passed to pylint."
parser = argparse.ArgumentParser(description=desc, allow_abbrev=False)
parser.add_argument('--fail-under', dest='threshold', type=float, default=8,
help='If the final score is more than THRESHOLD, exit with'
' exitcode 0, and pylint's exitcode otherwise.')
args, remaining_args = parser.parse_known_args()
threshold = args.threshold
run = lint.Run(remaining_args, do_exit=False)
score = run.linter.stats['global_note']
if score < threshold:
sys.exit(run.linter.msg_status)
As other answers here have mentioned, pylint.lint.Run is the way to go, however, there is not a good demonstration of Python 3 specific uses. Also, pylint gives a useful reporter class that should be used to capture the output of a pylint run. There are options for all different output types. In the following examples, I’ll capture the Colorized output using the ColorizedTextReporter class.
As of pylint 2.12
from pylint import lint
from pylint.reporters import text
# list of pylint args to only enable warnings, errors, and fatals
args = ["my_file.py", "--enable=W,E,F"]
pylint_output = StringIO() # need a StringIO object where the output will be stored
reporter = text.ColorizedTextReporter(pylint_output)
run = lint.Run(args, reporter=reporter, exit=False) # exit=False means don't exit when the run is over
score = run.lint.stats.global_note # pylint score out of 10
# also available are stats.error, stats.warnings, etc...
Before pylint 2.12
The only difference is that the stats object of Run().lint is a dictionary. So you’d get the score by run.lint.stats.get("global_note")
To get pylint report in python script see an example
from pylint import lint
from pylint.reporters import CollectingReporter
from dataclasses import asdict
report = CollectingReporter()
result = lint.Run([TOP_DIR], reporter=report, do_exit=False)
print("result ",result.linter.stats.global_note)
line_format = "{path}:{line}:{column}: {msg_id}: {msg} ({symbol})"
for error in report.messages:
print(line_format.format(**asdict(error)))
Based on few solutions here, I came with the following (mixing StringIO to avoid creating a "simple class" when something already exists) and the TextReporter from pylint:
from io import StringIO
from pylint.lint import Run
from pylint.reporters.text import TextReporter
files = ["my_file.py"]
pylint_options = ["--disable=line-too-long,import-error,fixme"]
pylint_output = StringIO()
Run(
files + pylint_options,
reporter=TextReporter(pylint_output),
exit=False,
)
for line in pylint_output.getvalue().splitlines():
... # do something with each line, like filtering / extracting
You can also use ParseableTextReporter instead of TextReporter and the results will be slightly different (maybe easier to parse):
With TextReporter:
************* Module my_file
my_file.py:65:0: W0613: Unused argument 'args' (unused-argument)
my_file.py:65:0: W0613: Unused argument 'kwargs' (unused-argument)
my_file.py:90:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:93:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:96:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:99:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:102:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:183:0: C0116: Missing function or method docstring (missing-function-docstring)
------------------------------------------------------------------
Your code has been rated at 9.61/10 (previous run: 9.61/10, +0.00)
With ParseableTextReporter:
************* Module my_file
my_file.py:65: [W0613(unused-argument), CoverageSummaryTable.__init__] Unused argument 'args'
my_file.py:65: [W0613(unused-argument), CoverageSummaryTable.__init__] Unused argument 'kwargs'
my_file.py:90: [C0116(missing-function-docstring), ProtoLogger.debug] Missing function or method docstring
my_file.py:93: [C0116(missing-function-docstring), ProtoLogger.info] Missing function or method docstring
my_file.py:96: [C0116(missing-function-docstring), ProtoLogger.warning] Missing function or method docstring
my_file.py:99: [C0116(missing-function-docstring), ProtoLogger.error] Missing function or method docstring
my_file.py:102: [C0116(missing-function-docstring), ProtoLogger.critical] Missing function or method docstring
my_file.py:183: [C0116(missing-function-docstring), coverage_output] Missing function or method docstring
------------------------------------------------------------------
Your code has been rated at 9.61/10 (previous run: 9.61/10, +0.00)
It’s up to you 🙂
Thanks to @mad7777, @Amit Tripathi, and @mcarans
I’d like to invoke the Pylint checker, limited to the error signalling part, as part of my unit testing. So I checked the Pylint executable script, got to the pylint.lint.Run helper class and there I got lost in a quite long __init__ function, ending with a call to sys.exit().
Anybody ever tried and managed to do so?
The dream-plan would be this:
if __name__ == '__main__':
import pylint.lint
pylint.lint.something(__file__, justerrors=True)
# now continue with unit testing
Any hints? Other than "copy the __init__ method and skip the sys.exit()", I mean?
I don’t need the tests to be run by Pylint, it might as well be pyflakes or other software: feel free to suggest alternatives.
Take a look at the pylint/epylint.py file which contains two different ways to start Pylint programmatically.
You can also simply call
from pylint.lint import Run
Run(['--errors-only', 'myfile.py'])
for instance.
I’m glad I came across this. I used some of the answers here and some initiative to come up with:
# a simple class with a write method
class WritableObject:
def __init__(self):
self.content = []
def write(self, string):
self.content.append(string)
pylint_output = WritableObject()
pylint = lint.Run(args, reporter=ParseableTextReporter(pylint_output), exit=False)
Args in the above is a list of strings eg. [“-r”, “n”, “myfile.py”]
I got the same problem recently.
syt is right, pylint.epylint got several methods in there. However they all call a subprocess in which python is launched again. In my case, this was getting quite slow.
Building from mcarans answer, and finding that there is a flag exit, I did the following
class WritableObject(object):
"dummy output stream for pylint"
def __init__(self):
self.content = []
def write(self, st):
"dummy write"
self.content.append(st)
def read(self):
"dummy read"
return self.content
def run_pylint(filename):
"run pylint on the given file"
from pylint import lint
from pylint.reporters.text import TextReporter
ARGS = ["-r","n", "--rcfile=rcpylint"] # put your own here
pylint_output = WritableObject()
lint.Run([filename]+ARGS, reporter=TextReporter(pylint_output), exit=False)
for l in pylint_output.read():
do what ever you want with l...
which is about 3 times faster in my case.
With this I have been going through a whole project, using full output to check each source file, point errors, and rank all files from their note.
Instead of creating a WritableObject class we can use StringIO. StringIO contains write method.
import sys
try:
from io import StringIO
except:
from StringIO import StringIO
stdout = sys.stdout
sys.stdout = StringIO()
ARGS = ["-r","n", "--rcfile=rcpylint"]
r = lint.Run(['../test.py']+ARGS, exit=False)
test = sys.stdout.getvalue()
sys.stdout.close()
sys.stdout = stdout
print (test.split('n'))
Source:
Another entry point for pylint is the epylint.py_run function, that implement the stdout and stderr interception. However, as shown in the following code, pylint seems to not write its reports in stdout:
from pylint import epylint
pylint_stdout, pylint_stderr = epylint.py_run(__file__, return_std=True)
print(pylint_stdout.getvalue()) # -> there is just the final rank, no report nor message
print(pylint_stderr.getvalue())
Now, i found that pylint from API and pylint from CLI do not use the same default parameters. So, you just have to provides the parameters you need to pylint.
from pylint import epylint
options = '--enable=all' # all messages will be shown
options += '--reports=y' # also print the reports (ascii tables at the end)
pylint_stdout, pylint_stderr = epylint.py_run(__file__ + ' ' + options, return_std=True)
print(pylint_stdout.getvalue())
print(pylint_stderr.getvalue())
As described here, pylint will perform the parsing itself, and will correctly output the expected results in stdout.
NOTE:
At some point pylint changed the interface. The above examples need to replace exit=False with do_exit=False. (@mad7777, @amit-tripathi)
(Learned as per https://github.com/carsongee/pytest-pylint/issues/80)
Here is a wrapper I use to programmatically call pylint so I have a –fail-under arg to overwite the default pylint exit code (usefull for CI). This snippet was tested using pylint 2.3.1
""" Execute pylint and fail if score not reached. """
import argparse
import sys
from pylint import lint
desc = "PyLint wrapper that add the --fail-under option."
" All other arguments are passed to pylint."
parser = argparse.ArgumentParser(description=desc, allow_abbrev=False)
parser.add_argument('--fail-under', dest='threshold', type=float, default=8,
help='If the final score is more than THRESHOLD, exit with'
' exitcode 0, and pylint's exitcode otherwise.')
args, remaining_args = parser.parse_known_args()
threshold = args.threshold
run = lint.Run(remaining_args, do_exit=False)
score = run.linter.stats['global_note']
if score < threshold:
sys.exit(run.linter.msg_status)
As other answers here have mentioned, pylint.lint.Run is the way to go, however, there is not a good demonstration of Python 3 specific uses. Also, pylint gives a useful reporter class that should be used to capture the output of a pylint run. There are options for all different output types. In the following examples, I’ll capture the Colorized output using the ColorizedTextReporter class.
As of pylint 2.12
from pylint import lint
from pylint.reporters import text
# list of pylint args to only enable warnings, errors, and fatals
args = ["my_file.py", "--enable=W,E,F"]
pylint_output = StringIO() # need a StringIO object where the output will be stored
reporter = text.ColorizedTextReporter(pylint_output)
run = lint.Run(args, reporter=reporter, exit=False) # exit=False means don't exit when the run is over
score = run.lint.stats.global_note # pylint score out of 10
# also available are stats.error, stats.warnings, etc...
Before pylint 2.12
The only difference is that the stats object of Run().lint is a dictionary. So you’d get the score by run.lint.stats.get("global_note")
To get pylint report in python script see an example
from pylint import lint
from pylint.reporters import CollectingReporter
from dataclasses import asdict
report = CollectingReporter()
result = lint.Run([TOP_DIR], reporter=report, do_exit=False)
print("result ",result.linter.stats.global_note)
line_format = "{path}:{line}:{column}: {msg_id}: {msg} ({symbol})"
for error in report.messages:
print(line_format.format(**asdict(error)))
Based on few solutions here, I came with the following (mixing StringIO to avoid creating a "simple class" when something already exists) and the TextReporter from pylint:
from io import StringIO
from pylint.lint import Run
from pylint.reporters.text import TextReporter
files = ["my_file.py"]
pylint_options = ["--disable=line-too-long,import-error,fixme"]
pylint_output = StringIO()
Run(
files + pylint_options,
reporter=TextReporter(pylint_output),
exit=False,
)
for line in pylint_output.getvalue().splitlines():
... # do something with each line, like filtering / extracting
You can also use ParseableTextReporter instead of TextReporter and the results will be slightly different (maybe easier to parse):
With TextReporter:
************* Module my_file
my_file.py:65:0: W0613: Unused argument 'args' (unused-argument)
my_file.py:65:0: W0613: Unused argument 'kwargs' (unused-argument)
my_file.py:90:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:93:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:96:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:99:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:102:4: C0116: Missing function or method docstring (missing-function-docstring)
my_file.py:183:0: C0116: Missing function or method docstring (missing-function-docstring)
------------------------------------------------------------------
Your code has been rated at 9.61/10 (previous run: 9.61/10, +0.00)
With ParseableTextReporter:
************* Module my_file
my_file.py:65: [W0613(unused-argument), CoverageSummaryTable.__init__] Unused argument 'args'
my_file.py:65: [W0613(unused-argument), CoverageSummaryTable.__init__] Unused argument 'kwargs'
my_file.py:90: [C0116(missing-function-docstring), ProtoLogger.debug] Missing function or method docstring
my_file.py:93: [C0116(missing-function-docstring), ProtoLogger.info] Missing function or method docstring
my_file.py:96: [C0116(missing-function-docstring), ProtoLogger.warning] Missing function or method docstring
my_file.py:99: [C0116(missing-function-docstring), ProtoLogger.error] Missing function or method docstring
my_file.py:102: [C0116(missing-function-docstring), ProtoLogger.critical] Missing function or method docstring
my_file.py:183: [C0116(missing-function-docstring), coverage_output] Missing function or method docstring
------------------------------------------------------------------
Your code has been rated at 9.61/10 (previous run: 9.61/10, +0.00)
It’s up to you 🙂
Thanks to @mad7777, @Amit Tripathi, and @mcarans