Make black[d] install + test run with 3.12

[cooperlees]

• With aiohttp >= 3.9.0 we can now install all dependencies with 3.12
• Add actions to run 3.12
• Lint still needs to run in 3.11 ... Will create an issue / workout how to fix that in another PR
  • Really want to get our 3.12 tests running on PRs so we don't accidentally regress is my main goal here

Test:

• python3.12 -m venv /tmp/tb --upgrade-deps
• /tmp/tb/bin/pip install tox
• /tmp/tb/bin/pip install .[d]
• /tmp/tb/bin/tox -e py312

  py312: OK (37.61=setup[3.98]+cmd[3.83,0.36,19.54,6.46,3.00,0.44] seconds)
  congratulations :) (37.63 seconds)

[cooperlees]

mypy pre-commit mirror uses aiohttp and fails in 3.12:

[INFO] Initializing environment for https://github.com/pre-commit/mirrors-mypy:types-PyYAML,tomli >= 0.2.6, < 2.0.0,click >= 8.1.0, != 8.1.4, != 8.1.5,packaging >= 22.0,platformdirs >= 2.1.0,pytest,hypothesis,aiohttp >= 3.7.4,types-commonmark,urllib3,hypothesmith.

Haven't looked for an issue etc. yet - Will check again once a real 3.9.0 comes out ... was helping test the beta for aiohttp.

[JelleZijlstra]

See #4050 for the PyPy Windows failure.

[cooperlees]

See #4050 for the PyPy Windows failure.

Hah - was just researching this ...

Should we run newer pypy's now too? Wonder if GitHub as support for pypy3.9 + 3.10 https://www.pypy.org/download.html

[JelleZijlstra]

I feel like one PyPy is enough CI to deal with :) It tends to be one of the slowest steps.

[cooperlees]

I feel like one PyPy is enough CI to deal with :) It tends to be one of the slowest steps.

Agree - so what if we move to only the latest pypy? That is pypy3.10 today ... Am I reading correctly we could use latest_pypy as well?

[JelleZijlstra]

It's generally good to use the earliest still-supported version because it has the most chance of oddities. I don't know what PyPy version people generally use, though.

[AlexWaygood]

It's generally good to use the earliest still-supported version because it has the most chance of oddities. I don't know what PyPy version people generally use, though.

According to https://foss.heptapod.net/pypy/pypy/-/issues/3958, only PyPy 3.9+ is considered supported by the PyPy project

[cooperlees]

See, I feel the opposite here. If you're using PyPy, you're generally already making a bespoke / custom environment to live in a PyPy world. Due to this upgrading to a later version happens more frequently and is generally easier for these environments to do so. With this behavior, I don't see the advantage of us trying to support years old PyPy. I do get tho that if it runs on the oldest supported version, it will probably (not as optimally cause we keep more tech debt around) run on the latest PyPy.

But I'll go with the consensus here ... I'm just sad at how long we've claimed 3.12 support but do not run any tests ... I think I'll look at making dedicated blackd CI after this so we reduce the footprint our aiohttp dependency has on our CI running.

[cooperlees]

WIll wait for aio-libs/aiohttp#7848 - But I doubt it will be a quick fix + release ...

[JelleZijlstra]

I'd be OK with running PyPy 3.10 if you think that's enough. I don't have a great sense of what people who use PyPy to run Black would actually want. If we break PyPy 3.9 and people complain, we can always change it back.

[cooperlees]

Anyone else understand the diff-shades preview fail?

[JelleZijlstra]

It's been failing randomly for some time. Unfortunately we don't have any clear leads. It could be excessive resource usage of some kind.

[cooperlees]

Yeah, looks like that with being unable to cleanup:

/opt/hostedtoolcache/Python/3.11.6/x64/lib/python3.11/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
Error: The operation was canceled.

Wonder if we can request a larger runner for diff-shades ... Like I want to do for docker build too ...