Lazy loading conflicts with modules overriding ModuleType

[effigies]

Bug report

Bug description:

Discovered while working on #117178.

PEP 726 notes that the existing way for a module to override default methods is:

# mod.py

import sys
from types import ModuleType

CONSTANT = 3.14

class ImmutableModule(ModuleType):
    def __setattr__(name, value):
        raise AttributeError('Read-only attribute!')

    def __delattr__(name):
        raise AttributeError('Read-only attribute!')

sys.modules[__name__].__class__ = ImmutableModule

If I want to load this module lazily, I can use this MRE (adapted from #117178):

import sys
import importlib
import importlib.util

def lazy_import(name):
    spec = importlib.util.find_spec(name)
    loader = importlib.util.LazyLoader(spec.loader)
    spec.loader = loader
    module = importlib.util.module_from_spec(spec)
    sys.modules[name] = module
    loader.exec_module(module)
    return module

# Eagerly load the module
eager_module = importlib.import_module(sys.argv[1])
print(f"Eagerly loaded: {type(eager_module)}")
del sys.modules[sys.argv[1]]

# Lazy-load the module...
lazy_module = lazy_import(sys.argv[1])
print(f"Lazy-loaded: {type(lazy_module)}")
# ... and then trigger load by listing its contents
print(dir(lazy_module)[:5])
print(f"Fully loaded: {type(lazy_module)}")

This currently (with the fix in #117179) produces:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
Traceback (most recent call last):
  File "/home/chris/Projects/cpython/repro.py", line 23, in <module>
    print(dir(lazy_module)[:5])
          ~~~^^^^^^^^^^^^^
  File "/home/chris/Projects/cpython/Lib/importlib/util.py", line 223, in __getattribute__
    self.__class__ = types.ModuleType
    ^^^^^^^^^^^^^^
  File "/home/chris/Projects/cpython/mod.py", line 8, in __setattr__
    raise AttributeError('Read-only attribute!')
AttributeError: Read-only attribute!. Did you mean: '__doc__'?

Making the following change:

-                self.__class__ = types.ModuleType
+                object.__setattr__(self, '__class__', types.ModuleType)

Results in:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
['CONSTANT', 'ImmutableModule', 'ModuleType', '__builtins__', '__cached__']
Fully loaded: <class 'module'>

If instead we change to:

-                self.__class__ = types.ModuleType
+                if isinstance(self, _LazyModule):
+                    object.__setattr__(self, '__class__', types.ModuleType)

We get:

❯ ./python repro.py mod
Eagerly loaded: <class 'mod.ImmutableModule'>
Lazy-loaded: <class 'importlib.util._LazyModule'>
['CONSTANT', 'ImmutableModule', 'ModuleType', '__builtins__', '__cached__']
Fully loaded: <class 'mod.ImmutableModule'>

I believe making this change is the best way to respect the intentions of the module writer.

I've tagged this as a bug because these things have not worked together, but it might be more of a feature for them to start to, as neither is exactly standard practice.

CPython versions tested on:

3.11, 3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-117182: Allow lazily loaded modules to modify their own __class__ #117185