[doc pre-commit integration] Details about known caveats of using mul…

…tiple processes

Refs microsoft/vscode-pylint#454

Closes #9341

.pre-commit-config.yaml

@@ -77,6 +77,9 @@ repos:
         entry: pylint
         language: system
         types: [python]
+        # Not that problematic to run in parallel see Pre-commit
+        # integration in the doc for details
+        # require_serial: true
         args: ["-rn", "-sn", "--rcfile=pylintrc", "--fail-on=I"]
         exclude: tests(/\w*)*/functional/|tests/input|tests(/\w*)*data/|doc/
       # We define an additional manual step to allow running pylint with a spelling

doc/faq.rst

@@ -67,6 +67,22 @@ to not be included as default messages.
 You can see the plugin you need to explicitly :ref:`load in the technical reference
 <user_guide/checkers/extensions:optional checkers>`.
 
+I want to use pylint on save in my IDE, how can I do that ?
+-----------------------------------------------------------
+
+Don't do it, pylint is not fast enough for that and never will be. pylint is best suited
+for a continuous integration job or a git ``pre-push`` hook, especially if your repository
+is large.
+
+Why do I have non deterministic result when I try to parallelize pylint ?
+-------------------------------------------------------------------------
+
+pylint should analyses all your code at once in order to best infer the
+actual values that are passed in calls. If only some of the files are given pylint might
+miss a particular value's type and produce inferior inference for the subset. Parallelization
+of pylint is not easy, we also discourage the use of the ``-j`` option if this matter to you.
+
+
 Which messages should I disable to avoid duplicates if I use other popular linters ?
 ------------------------------------------------------------------------------------
 

doc/user_guide/installation/pre-commit-integration.rst

@@ -3,10 +3,28 @@
 Pre-commit integration
 ======================
 
-``pylint`` can be used as a `pre-commit <https://pre-commit.com>`_ hook.
+``pylint`` can be used as a `pre-commit <https://pre-commit.com>`_ hook. We however
+discourage it as pylint -- due to its speed -- is more suited to a continuous integration
+job or a git ``pre-push`` hook, especially if your repository is large.
 
 Since ``pylint`` needs to import modules and dependencies to work correctly, the
-hook only works with a local installation of ``pylint`` (in your environment).
+hook only works with a local installation of ``pylint`` (in your environment). It means
+it can't be used with ``pre-commit.ci`` and you will need to add the following to your
+``.pre-commit-config.yaml`` ::
+
+.. sourcecode:: yaml
+
+    ci:
+      skip: [pylint]
+
+Another limitation is that pylint should analyses all your code at once in order to best infer the
+actual values that are passed in calls. If only some of the files are given pylint might
+miss a particular value's type and produce inferior inference for the subset. Since pre-commit slice
+the files given to it in order to parallelize the processing the result can be degraded.
+It can also be unexpectedly different when the file set changes because the new slicing can change
+the inference. Thus the ``require_serial`` option could be set to ``true`` if correctness and determinism
+is more important than parallelization to you.
+
 If you installed ``pylint`` locally it can be added to ``.pre-commit-config.yaml``
 as follows:
 
@@ -19,6 +37,7 @@ as follows:
         entry: pylint
         language: system
         types: [python]
+        require_serial: true
         args:
           [
             "-rn", # Only display messages