Invoking pants in CI

[jake-normal]

It is often desirable to write CI scripts in a "real programming language" like Python. Obviously, those scripts live in our monorepo, and at that point they might as well have access to all the other Python libraries in the repo, meaning they should be managed by Pants. Since they are typically simple scripts, we tend to invoke them in CI with Pants, like this:

pants run deploy/deploy.py

Since these are CI scripts, they often need answers to questions about the code in the repo. Fortunately, Pants has helpful introspection features, and so these scripts often invoke Pants' introspection commands. Now, you effectively have Pants invoking Pants! And of course Pants only likes one instance of Pants running at any given time, though we can set concurrent to True to work around that.

Does anything about this pattern seem weird to you? E.g. would it be better to create Python packages for these CI tools, build them, and then invoke them from the dist folder? Or is the pattern that we are following perfectly normal?

[benjyw]

I think there is precedent for this in Pants's own CI, e.g., https://github.com/pantsbuild/pants/blob/73d41b4c9935d18a2aa94c3bff6d8f99e0cc0dcb/src/python/pants_release/release.py

However note that you do lose the advantage of pantsd on that inner invocation.

[jake-normal]

Interesting. Okay, good to know that there isn't a better pattern that we are missing.