TypeError in importlib.metadata when distribution metadata has no name

[jack9603301]

Bug report

A clear and concise description of what the bug is.
Include a minimal, reproducible example (https://stackoverflow.com/help/minimal-reproducible-example), if possible.

I don't know how to describe this bug, see

The problem is with the following code:

cpython/Lib/importlib/metadata/__init__.py

Lines 866 to 872 in 95c55a6

	@staticmethod
	def normalize(name):
	"""
	PEP 503 normalization plus dashes as underscores.
	"""
	return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')

    @staticmethod
    def normalize(name):
        """
        PEP 503 normalization plus dashes as underscores.
        """
        return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')

The fixes are as follows:

Guess the reason is that in some cases, name may be a None, if no judgment is made, an exception will always be thrown under certain conditions, which causes an exception in my gentoo regardless of any python3.10 library installed

Your environment

gentoo/Linux

[ericvsmith]

Please show us how you caused this exception so that we can reproduce it.

[jack9603301]

It is in my gentoo, there will be problems when using emerge to install py components, it may take a little time to trigger with py script

[jaraco]

What's happening here is that some Distribution as discovered in the user's environment has a value of None for the name, which is disallowed. Every distribution should have at least a name and version. Related is an effort to avoid returning None for missing metadata values (python/importlib_metadata#371), but the current behavior is that if a Distribution exists but it has no metadata, the metadata will have None values.

I don't believe we want to be more lenient to broken metadata here, but instead provide the user better error messages about the invalid metadata.

I can replicate the error by creating an invalid metadata path:

$ mkdir pkg-info
$ py -3.10 -c "import importlib.metadata as md, pathlib; dist = md.PathDistribution(pathlib.Path('pkg-info')); print(dist._normalized_name)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/metadata/__init__.py", line 943, in _normalized_name
    or super()._normalized_name
  File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/metadata/__init__.py", line 622, in _normalized_name
    return Prepared.normalize(self.name)
  File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/metadata/__init__.py", line 871, in normalize
    return re.sub(r"[-_.]+", "-", name).lower().replace('-', '_')
  File "/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/re.py", line 209, in sub
    return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object

@jack9603301 , it would really help to have more information about the problem. In particular, what version of Python are you running? What steps did you run to get the error (which package or packages did you install)?

I tried installing python with emerge on gentoo, but I'm not familiar enough with gentoo to know how to bootstrap an environment. I tried using the gentoo/portage docker image as it has the most stars on dockerhub, but it doesn't even have bash and it doesn't have emerge.

 draft $ docker run --platform linux/amd64 -it gentoo/portage sh
/ # emerge install python
sh: emerge: not found

I tried running with gentoo/stage-3, but it seems emerge isn't viable yet:

 draft $ docker run --platform linux/amd64 -it gentoo/stage3 emerge
Unable to find image 'gentoo/stage3:latest' locally
latest: Pulling from gentoo/stage3
5396afd91063: Pull complete 
Digest: sha256:d6f040d60958cbb68bb51b4c96dc1fbc764b805ec1907f1068a32fbf536b77cc
Status: Downloaded newer image for gentoo/stage3:latest
!!! Section 'gentoo' in repos.conf has location attribute set to nonexistent directory: '/var/db/repos/gentoo'
!!! Invalid Repository Location (not a dir): '/var/db/repos/gentoo'


!!! /etc/portage/make.profile is not a symlink and will probably prevent most merges.
!!! It should point into a profile within /var/db/repos/gentoo/profiles/
!!! (You can safely ignore this message when syncing. It's harmless.)


!!! Your current profile is invalid. If you have just changed your profile
!!! configuration, you should revert back to the previous configuration.
!!! Allowed actions are limited to --help, --info, --search, --sync, and
!!! --version.

Are you familiar with Docker? If so, can you create the steps in a docker environment that replicate the reported issue? If not, do you have any advice on how to get emerge working in a virgin gentoo environment?

[jack9603301]

@jaraco Let me see. Do you need me to use Docker to reproduce?

[jaraco]

I don't necessarily need Docker, although I do find that to be one of the easiest ways to recreate a Linux-based environment with its particular idiosyncrasies. I'd be just as happy with a set of instructions that replicates the issue in an Ubuntu (or maybe Gentoo) environment (I'm more familiar with Debian/Ubuntu).

[DanAndersen]

For anyone else out there who lost a few hours trying to track down what's going on with this error, here's a code snippet to identify which broken package is the culprit:

from importlib import metadata
ds = metadata.distributions()
for d in ds:
    print("===")
    print(d)
    print(f"{d.name=}")
    if d.name is None:
        print("FOUND")
        broken_dist = d
        print(f"found broken distribution at {d._path}")
        break

This helped me find an egg directory that was broken, so that I could reinstall the pip package.

[jaraco]

In python/importlib_metadata#371, a related issue is being addressed. Once that issue is addressed, the call to retrieve the name will throw a KeyError, providing better clarity on the cause of the issue (a missing name versus a type error later). That, along with the diagnose routine, should make it easier for users to detect and diagnose issues like these.