ValueError: invalid file open mode 'U' on Python 3.11 #2401

mock · 2023-01-18T15:31:45Z

Description

I'm logging this here for reference per the advice given by those in the IRC support channel.

After upgrading my OS, my default Python is now 3.11.1. When I tried to start up my bots, I received multiple errors for the my custom modules. The error manifested as:

[2023-01-18 14:24:24,238] sopel.bot            ERROR    - Error loading weather: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'

The full error file is included below for a complete trace.

It was suggested for the time being to pull a copy of the master branch and then make any adjustments to the code to avoid the 'U' mode. (The line causing the problems was located here: https://github.com/sopel-irc/sopel/blob/v7.1.9/sopel/plugins/handlers.py#L435.)

Reproduction steps

Upgrade/Install Fedora 37
pip install --upgrade sopel
sopel start -c bot.cfg -d

Expected behavior

Loading of my custom modules as with Python 3.10.x. Custom modules then will listen for triggers.

Relevant logs

[2023-01-18 09:16:02,039] sopel.irc            ERROR    - Fatal error in core, please review exceptions log.
[2023-01-18 09:16:01,492] sopel.irc.backends   ERROR    - Server timeout detected after 550.133971s; closing.
[2023-01-18 09:16:23,122] sopel.bot            ERROR    - Cannot register URL callback: Bugzilla URL callback requires "bugzilla.domains" to be configured; check your config file.
[2023-01-18 14:24:22,565] sopel.bot            ERROR    - Cannot register URL callback: Bugzilla URL callback requires "bugzilla.domains" to be configured; check your config file.
[2023-01-18 14:24:22,717] sopel.bot            ERROR    - Error in help setup: global flags not at the start of the expression at position 14
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 337, in setup_plugins
    plugin.register(self)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 330, in register
    bot.add_plugin(self, *relevant_parts)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 479, in add_plugin
    self.register_callables(callables)
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 620, in register_callables
    rule = plugin_rules.Rule.from_callable(settings, callbl)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 815, in from_callable
    regexes = cls.regex_from_callable(settings, handler)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 791, in regex_from_callable
    return tuple(_clean_rules(
           ^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 87, in _clean_rules
    yield _compile_pattern(pattern, nick, aliases)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/rules.py", line 106, in _compile_pattern
    return re.compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/__init__.py", line 227, in compile
    return _compile(pattern, flags)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/__init__.py", line 294, in _compile
    p = _compiler.compile(pattern, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_compiler.py", line 743, in compile
    p = _parser.parse(p, flags)
        ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 980, in parse
    p = _parse_sub(source, state, flags & SRE_FLAG_VERBOSE, 0)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 455, in _parse_sub
    itemsappend(_parse(source, state, verbose, nested + 1,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/re/_parser.py", line 841, in _parse
    raise source.error('global flags not at the start '
re.error: global flags not at the start of the expression at position 14
[2023-01-18 14:24:24,221] sopel.bot            ERROR    - Error loading buzz_words: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,224] sopel.bot            ERROR    - Error loading camel: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,227] sopel.bot            ERROR    - Error loading hello_world: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,230] sopel.bot            ERROR    - Error loading test: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,233] sopel.bot            ERROR    - Error loading rick: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,235] sopel.bot            ERROR    - Error loading wthr: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'
[2023-01-18 14:24:24,238] sopel.bot            ERROR    - Error loading weather: invalid file open mode 'U'
Traceback (most recent call last):
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/bot.py", line 323, in setup_plugins
    plugin.load()
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 471, in load
    self._module = self._load()
                   ^^^^^^^^^^^^
  File "/home/mock/.local/lib/python3.11/site-packages/sopel/plugins/handlers.py", line 436, in _load
    mod = imp.load_module(self.name, mod, self.path, description)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/imp.py", line 230, in load_module
    raise ValueError('invalid file open mode {!r}'.format(mode))
ValueError: invalid file open mode 'U'

Notes

No response

Sopel version

7.1.9

Installation method

pip install

Python version

3.11.1

Operating system

Fedora 37

IRCd

No response

Relevant plugins

all my custom plugins

The text was updated successfully, but these errors were encountered:

SnoopJ · 2023-01-18T15:35:04Z

File mode 'U' was removed in 3.11, the code responsible for this error was dropped in 596adc4 but the change isn't yet in a release. Users who run into the problem should install Sopel from master for the time being, or maybe run their instance with 3.10 or lower until the next release.

Thanks for the report!

dgw · 2023-01-18T15:39:05Z

I've created and assigned a new milestone, with an arbitrarily assigned "pay attention to me!" date of 1 May. If we don't get 8.0 finalized by then, we can cut a hotfix release for the stable branch. (If the Travis CI release pipeline no longer works, I'll build and upload it myself!) Meanwhile, I will pin the issue.

Note: Hotfix urgency to be adjusted as needed if too many users start running into this, i.e. if Python 3.11 environments become too common in the wild to ignore.

mock · 2023-01-18T15:51:00Z

Just confirming that installing the master branch resolved the above issue without any other tweaking.

alethiophile · 2023-06-15T16:13:48Z

Just here to note that I ran into this error today on installing the latest release of sopel, which apparently is still not compatible with Python 3.11 despite this error being known for five months.

Just cut a release already, even if this is the only thing in it.

dgw · 2023-06-15T21:08:55Z

We can't run the maintenance branch's CI on GitHub Actions, and the old Travis-CI test + release pipeline is also broken due to changes from them (even though we registered as an OSS project 🙄)

Basically, creating a maintenance release that is adequately tested just for this bug (which has a workaround—installing from the master branch) would take too much of our limited developer time away from finishing Sopel 8.0, which is on track to be done next month. Seriously, we're very close! Prioritization is just very important for us, as a small project with few active contributors.

dgw · 2023-12-06T06:48:15Z

I'm writing the Sopel 8 upgrade guide, and @SnoopJ has put in a bunch of work on the changelog for 8.0.0. It's safe to say that this won't be fixed in a patch release for 7.1.x, so I'm re-tagging this for 8.0.0 and closing it (because it is, in fact, fixed).

mock added Bug Things to squish; generally used for issues Needs Triage Issues that need to be reviewed and categorized labels Jan 18, 2023

dgw added this to the 7.1.10 milestone Jan 18, 2023

dgw added High Priority and removed Needs Triage Issues that need to be reviewed and categorized labels Jan 18, 2023

dgw pinned this issue Jan 18, 2023

dgw changed the title ~~ValueError: invalid file open mode 'U'~~ ValueError: invalid file open mode 'U' on Python 3.11 Jan 18, 2023

SnoopJ mentioned this issue Jun 15, 2023

Cherry-pick: Drop imp usage on 7.1.x branch #2474

Closed

4 tasks

dgw modified the milestones: 7.1.10, 8.0.0 Dec 6, 2023

dgw closed this as completed Dec 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ValueError: invalid file open mode 'U' on Python 3.11 #2401

ValueError: invalid file open mode 'U' on Python 3.11 #2401

mock commented Jan 18, 2023

SnoopJ commented Jan 18, 2023

dgw commented Jan 18, 2023

mock commented Jan 18, 2023

alethiophile commented Jun 15, 2023

dgw commented Jun 15, 2023

dgw commented Dec 6, 2023

ValueError: invalid file open mode 'U' on Python 3.11 #2401

ValueError: invalid file open mode 'U' on Python 3.11 #2401

Comments

mock commented Jan 18, 2023

Description

Reproduction steps

Expected behavior

Relevant logs

Notes

Sopel version

Installation method

Python version

Operating system

IRCd

Relevant plugins

SnoopJ commented Jan 18, 2023

dgw commented Jan 18, 2023

mock commented Jan 18, 2023

alethiophile commented Jun 15, 2023

dgw commented Jun 15, 2023

dgw commented Dec 6, 2023