blob: 64d4421a735a18c4f51fe44233f0cb1e7be37abc (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
|
Launching tests
===============
pytest
------
Since we use pytest_ to run the tests, you can also use it on its own.
We do recommend using the tox_ command though::
pytest pylint -k test_functional
You can use pytest_ directly. If you want to run tests on a specific portion of the
code with pytest_ and your local python version::
python3 -m pytest
Everything in tests/message with coverage for the relevant code (require `pytest-cov`_)::
python3 -m pytest tests/message/ --cov=pylint.message
coverage html
Only the functional test "missing_kwoa_py3"::
python3 -m pytest "tests/test_functional.py::test_functional[missing_kwoa_py3]"
tox
---
You can also *optionally* install tox_ and run our tests using the tox_ package, as in::
python -m tox
python -m tox -epy38 # for Python 3.8 suite only
python -m tox -epylint # for running Pylint over Pylint's codebase
python -m tox -eformatting # for running formatting checks over Pylint's codebase
It's usually a good idea to run tox_ with ``--recreate``. This flag tells tox_ to re-download
all dependencies before running the tests. This can be important when a new version of
astroid_ or any of the other dependencies has been published::
python -m tox --recreate # The entire tox environment will be recreated
python -m tox --recreate -e py310 # The python 3.10 tox environment will be recreated
To run only a specific test suite, use a pattern for the test filename
(**without** the ``.py`` extension), as in::
python -m tox -e py310 -- -k test_functional
python -m tox -e py310 -- -k \*func\*
python -m tox --recreate -e py310 -- -k test_functional # With recreation of the environment
.. _primer_tests:
Primer tests
------------
Pylint also uses what we refer to as ``primer`` tests. These are tests that are run automatically
in our Continuous Integration and check whether any changes in Pylint lead to crashes or fatal errors
on the ``stdlib``, and also assess a pull request's impact on the linting of a selection of external
repositories by posting the diff against ``pylint``'s current output as a comment.
To run the primer test for the ``stdlib``, which only checks for crashes and fatal errors, you can add
``--primer-stdlib`` to the pytest_ command. For example::
pytest -m primer_stdlib --primer-stdlib
To produce the output generated on Continuous Integration for the linting of external repositories,
run these commands::
python tests/primer/__main__.py prepare --clone
python tests/primer/__main__.py run --type=pr
To fully simulate the process on Continuous Integration, you should then checkout ``main``, and
then run these commands::
python tests/primer/__main__.py run --type=main
python tests/primer/__main__.py compare
The list of repositories is created on the basis of three criteria: 1) projects need to use a diverse
range of language features, 2) projects need to be well maintained and 3) projects should not have a codebase
that is too repetitive. This guarantees a good balance between speed of our CI and finding potential bugs.
You can find the latest list of repositories and any relevant code for these tests in the ``tests/primer``
directory.
.. _pytest-cov: https://pypi.org/project/pytest-cov/
.. _astroid: https://github.com/pylint-dev/astroid
|
