BUG: Handle .pyf.src and fix SciPy [urgent]

[HaoZeke]

Closes gh-25286. We don't have tests for .pyf.src files, which can be added in a follow up. It would also be nice for f2py to provide the from_template functionality instead of forcing generate_f2pymod.py in scipy to vendor it from distutils.

@tylerjereddy if this works for you as well I'd suggest it might be worthwhile to merge this now.

@charris this will need to be backported.

A lot of these are very annoyingly tiny bugs, but each one slowly inches f2py towards a better testsuite (among other things).

This PR:

• Don't lower .pyf files
• Stop making useless untitled modules
• Fix some of the #defines

Also:

• Vendors the .src file handling
• Re-establishes old distutils behavior w.r.t. compiled extension modules
  • Basically overwrites them

Tests:

• Ran scipy builds locally
• Added tests for both kinds of character handling (pre-ENH: Support character and character string arrays #19388 and post)

[HaoZeke]

Test failures are because of compute limits.

EDIT: manylinux 32-bit failure is real.

[HaoZeke]

Will finalize this tomorrow.

[tylerjereddy]

@tylerjereddy if this works for you as well I'd suggest it might be worthwhile to merge this now.

Ok, I skimmed through the diff and confirmed that SciPy main testsuite is back to "normal" (other unrelated issues) with this NumPy branch on M2 Mac, so seems fixed to me.

I'm obviously not an f2py expert, but there's some suggestion we should try to get this in and it does unbreak SciPy with some more regression tests.

[tylerjereddy]

I'll merge it, normally I squash backport candidates for SciPy to make backporting easier; however, maybe Chuck will forgive me for not squashing in this case given that some commits are distinct vendoring commits and so on, and it can also be helpful if we need to bisect again (hopefully not, but...).

[charris]

however, maybe Chuck will forgive me for not squashing

I plan to do worse, once f2py settles a bit I'm going to straight copy it down into 1.26.x. I'd usually let things ride, but 2.0 isn't going to come out this month. I some point I distutils might break in 1.26, but meson builds seem to be stabilizing downstream.

[rgommers]

Thanks for the quick fix!

It would also be nice for f2py to provide the from_template functionality instead of forcing generate_f2pymod.py in scipy to vendor it from distutils.

Please don't try this. I on purpose did not do that, not only because the .src format is unrelated to f2py but also because it's terrible and no one should start using it in 2023. The SciPy generate_f2pymod script is fine until we manage to get rid of this templating format completely.