replace pyupgrade+isort+autoflake with ruff

[jakkdl]

Take 3 after
master...use-ruff and #102

fixes #101

Notes:

• The one feature we're losing with replacing autoflake is not exploding star imports. (There's no test for that functionality.)
• when looking for tests I traversed git blame, so bfb380e is now ignored. Unfortunately the commit[s] when @Zac-HD added a ton of carriage returns are in the same commits as adding a bunch of other stuff, so can't ignore them :P
• ruff removing pass also removes unnecessary ..., hence change in pattern-matching.txt.

remaining TODOs:

• replace usage of autoflake.is_python_file
  • I can probably just inline it.
• replace black with ruff format (could leave that for a separate PR)
  • figure out a way to not rely on black for autodetecting target version
• figure out minimum ruff version
• open issue about git worktrees
  • I made the test xfail for now
• remove any now-redundant autofixers
  • ~fully done. I'm now working on documenting the remaining ones properly, partly because wtf-why-isn't-that-done-already, and partly so the ruff folks can consider implementing them. But that could be for a different PR.

[jakkdl]

libcst 1.2 dropped support for 3.8. Resolving it by generating deps.txt on 3.8, but could also do it by limiting libcst to <1.2. Ooooor we can drop 3.8 😉

[jakkdl]

ugh that broke 3.12 somehow. Seems like we might actually have to drop 3.8?

[Zac-HD]

Thanks again @jakkdl, this is looking great!

(especially the documentation so that ruff can pick up more of the codemods... extremely valuable and would have sat on my 'todo someday' list ~forever)

[Zac-HD]

I think it was more because I prefer strict validation unless there's a specific reason not to, and there's the 'str is an iterable of str' footgun.

[jakkdl]

ugh, right. I suppose I could change it to Iterable[str] and assert not isinstance(first_party_imports, str)?

[Zac-HD]

and then convert it to a frozenset? Sounds good.

[jakkdl]

It was previously passed to isort as a frozenset, but now we only use it as a parameter to ruff - where I currently do (the equivalent of) str(list(first_party_imports)) for formatting it. So if we'd only accept a list[str] we don't need to convert the type... not that it really matters though.

[Zac-HD]

Yeah, having a known list seems fine, and we can just take the union of stdlib modules in all non-EOL Python versions.

[jakkdl]

Oh, on >3.10 there's https://docs.python.org/3/library/sys.html#sys.stdlib_module_names so only need to generate static lists for 3.8 and 3.9

[jakkdl]

or well, depends on how thorough we want to be + the level of support for running-version-being-different-from-checked-version

[jakkdl]

https://pypi.org/project/stdlibs/ sounds pretty great if we want to be maximum thorough

[Zac-HD]

Let's just use that on 3.10+, and vendor the 3.10 list for older versions 👍

[jakkdl]

ugh that broke 3.12 somehow. Seems like we might actually have to drop 3.8?

Oh, can just add libcst<1.2; python_version < "3.9" to deps/check.in.

For funsies I also tried running 3.13 locally, but encountered HypothesisWorks/hypothesis#3916

Mostly unrelated to this PR:

Looking at https://github.com/Zac-HD/shed/actions/runs/8175453287/job/22352606988#step:5:75 says

git apply .hypothesis/patches/2024-03-06--c3414d12.patch to add failing examples to your code.

Not that it matters much here (I think? kinda weird error...), but is it possible to get at that patch in any way when it's run in CI? If not, does hypothesis have a flag for printing that patch in a different way to the log so a user could make use of it?

[jakkdl]

The CI check previously failed because tox -e check runs shed, auto-detecting all files in the git repository, which includes CODEMODS.md, and ruff then wants to modify the "input code" in two of the examples. I managed to get around it by using '''py instead of '''python (except with backticks) where ruff tries to butt in. Github still seems to display those correctly, but shed doesn't care for them (which it maybe should?)

[Zac-HD]

Mostly unrelated to this PR:

Looking at https://github.com/Zac-HD/shed/actions/runs/8175453287/job/22352606988#step:5:75 says

git apply .hypothesis/patches/2024-03-06--c3414d12.patch to add failing examples to your code.

Not that it matters much here (I think? kinda weird error...), but is it possible to get at that patch in any way when it's run in CI? If not, does hypothesis have a flag for printing that patch in a different way to the log so a user could make use of it?

If you've configured artifact upload for that directory, yes. It's useful to print them though, because if there's a printable example we already show them - in this case here with source_code='#\rµ',.

[Zac-HD]

I think this is a nice improvement over the status quo, so merging now.

If you want to make a follow-up PR with the stdlib list and optionally the iterable/frozenset handling, plus a new version number, I'll release shortly 🎉