running Glasgow exectutable produces "TypeError: expected string or bytes-like object" in packaging\utils.py #422

txf- · 2023-10-04T18:39:15Z

The example below is a build command but even just running Glasgow without any arguments, produces the same error. I updated to commit 62e27d9. This is still on python 3.9.

PS C:\Users\Tiago\Documents\Git> glasgow build --rev C3 uart
Traceback (most recent call last):
  File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\users\tiago\.local\bin\glasgow.exe\__main__.py", line 7, in <module>
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 910, in main
    exit(loop.run_until_complete(_main()))
  File "c:\python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 490, in _main
    args = get_argparser().parse_args()
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 283, in get_argparser
    add_applet_arg(p_run, mode="interact", required=True)
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 128, in add_applet_arg
    for handle, metadata in GlasgowAppletMetadata.all().items():
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 87, in all
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 87, in <dictcomp>
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 97, in __init__
    self.requirements = _requirements_for_optional_dependencies(
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 31, in _requirements_for_optional_dependencies
    if requirement.marker and requirement.marker.evaluate({"extra": dependency}):
  File "C:\Users\Tiago\.local\pipx\venvs\glasgow\lib\site-packages\packaging\markers.py", line 252, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "C:\Users\Tiago\.local\pipx\venvs\glasgow\lib\site-packages\packaging\markers.py", line 157, in _evaluate_markers
    lhs_value, rhs_value = _normalize(lhs_value, rhs_value, key=environment_key)
  File "C:\Users\Tiago\.local\pipx\venvs\glasgow\lib\site-packages\packaging\markers.py", line 131, in _normalize
    return tuple(canonicalize_name(v) for v in values)
  File "C:\Users\Tiago\.local\pipx\venvs\glasgow\lib\site-packages\packaging\markers.py", line 131, in <genexpr>
    return tuple(canonicalize_name(v) for v in values)
  File "C:\Users\Tiago\.local\pipx\venvs\glasgow\lib\site-packages\packaging\utils.py", line 47, in canonicalize_name
    value = _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object

I did try forcing it to be string in plugin.py:
if requirement.marker and requirement.marker.evaluate({"extra": str(dependency)}):

but after doing so I get an issue with the aiohttp dependency (so I don't know if this was the right thing to do):

  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\applet\audio\yamaha_opx\__init__.py", line 143, in <module>
    import aiohttp, aiohttp.web
ModuleNotFoundError: No module named 'aiohttp'

Here is the full trace:

PS C:\Users\Tiago\Documents\Git> glasgow.exe --version
Traceback (most recent call last):
  File "c:\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "c:\users\tiago\.local\bin\glasgow.exe\__main__.py", line 7, in <module>
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 910, in main
    exit(loop.run_until_complete(_main()))
  File "c:\python39\lib\asyncio\base_events.py", line 642, in run_until_complete
    return future.result()
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 490, in _main
    args = get_argparser().parse_args()
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 283, in get_argparser
    add_applet_arg(p_run, mode="interact", required=True)
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\cli.py", line 128, in add_applet_arg
    for handle, metadata in GlasgowAppletMetadata.all().items():
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 87, in all
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 87, in <dictcomp>
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\support\plugin.py", line 103, in __init__
    self._cls = entry_point.load()
  File "c:\python39\lib\importlib\metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "c:\python39\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Users\Tiago\Documents\Git\glasgow\software\glasgow\applet\audio\yamaha_opx\__init__.py", line 143, in <module>
    import aiohttp, aiohttp.web
ModuleNotFoundError: No module named 'aiohttp'

The text was updated successfully, but these errors were encountered:

whitequark · 2023-10-04T19:02:09Z

Try again on commit e4fd2dc?

txf- · 2023-10-04T20:11:12Z

Try again on commit e4fd2dc?

Same issue as above.

whitequark · 2023-10-04T20:14:51Z

What's your packaging version?

txf- · 2023-10-04T20:18:29Z

In the venv packaging==23.2
At the global level packaging==21.3

whitequark · 2023-10-04T21:26:10Z

@txf- Does it work fine if you revert 69d23bf? I've not been able to reproduce it so far.

txf- · 2023-10-04T22:20:31Z

@txf- Does it work fine if you revert 69d23bf?

It does not. Same issue.

ewenmcneill · 2023-10-07T09:06:06Z

I also ran into this on Python 3.9, this time on Ubuntu 20.04 LTS. But only found this GitHub issue when someone pointed it out.

After a bunch of debugging, I found I could work around the TypeError by coercing the entry_point.extras value from the re.Match() that was being returned, to a string, and then it worked fine. My fix is very brute force, so I don't know if you want it upstream. But maybe it inspires a solution.

FTR, I did try fixing it in _entry_points(), but unfortunately entry_point.extras is immutable, and I couldn't quickly figure out where importlib.metadata.distributions() was even getting that re.Match() from. Hence the kludge in the patch below.

From notes in #glasgow:

(Pdb) p entry_point.name, entry_point.extras
('audio-yamaha-opx', [<re.Match object; span=(1, 5), match='http'>])
seems to be the trigger.  And audio-yamaha-opx has a [http] suffix on its pyproject.toml.

and there's a gist of my original traceback -- https://gist.github.com/ewenmcneill/c30b58a83e4768e537eaa35f2b74829e -- which at a glance looks the same as the one at the top of this issue, but within pdb with some of my "what entry triggers this" state debugging.

Presumably this is a Python 3.9 importlib bug of some kind. But I suspect while Python 3.9 support is being kept (another year or so?) it'll probably have to be worked around somehow.

FTR, I too had the aiohttp dependency issue. Which I fixed with:

pipx3.9 inject glasgow aiohttp

and then later when I did the a reinstall to see if that fixed the issue, with:

pipx3.9 install -e --verbose --spec 'glasgow/software[builtin-toolchain,http]' glasgow

(note that the older pipx, eg, in Ubuntu 20.04 LTS, does need that --spec .... glasgow approach as its auto-detection of the package name from the "URL" otherwise fails badly, leading to it installing the venv, not finding the weirdly named package, and deleting the whole venv again :-/

Ewen

ewen@parthenon:/src/glasgow/software/glasgow/support$ git diff --cached
diff --git a/software/glasgow/support/plugin.py b/software/glasgow/support/plugin.py
index 8ef6631..8548bac 100644
--- a/software/glasgow/support/plugin.py
+++ b/software/glasgow/support/plugin.py
@@ -27,6 +27,10 @@ def _requirements_for_optional_dependencies(distribution, depencencies):
     requirements = map(packaging.requirements.Requirement, distribution.requires)
     selected_requirements = set()
     for dependency in depencencies:
+        import re
+        if isinstance(dependency, re.Match):
+            dependency = dependency.group(0)
+
         for requirement in requirements:
             if requirement.marker and requirement.marker.evaluate({"extra": dependency}):
                 requirement = packaging.requirements.Requirement(str(requirement))
ewen@parthenon:/src/glasgow/software/glasgow/support$

txf- · 2023-10-07T09:14:43Z

I can confirm that the above fix works to fix this specific issue, glasgow --version now works again. glasgow build --rev C3 uart does not, but that is a bug which is probably windows specific. I'll open a new report for that.

ewenmcneill · 2023-10-07T09:20:38Z

FTR, this seems to be the importlib being loaded for me:

ewen@parthenon:/usr/local/share/pipx/venvs/glasgow$ . bin/activate
(glasgow) ewen@parthenon:/usr/local/share/pipx/venvs/glasgow$ python3.9
python3.9         python3.9-config  
(glasgow) ewen@parthenon:/usr/local/share/pipx/venvs/glasgow$ python3.9
Python 3.9.5 (default, Nov 23 2021, 15:27:38) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import importlib.metadata
>>> print(importlib.metadata.__file__)
/usr/lib/python3.9/importlib/metadata.py
>>>

so it seems to be part of the Python 3.9 upstream distribution.

Ewen

ewenmcneill · 2023-10-07T09:26:15Z

Aha, there's a regex-over-a-regex-match to make the extras property:

    @property
    def extras(self):
        match = self.pattern.match(self.value)
        return re.findall(r'\w+', match.group('extras') or '')

at https://github.com/python/cpython/blob/43a6e4fa4934fcc0cbd83f7f3dc1b23a5f79f24b/Lib/importlib/metadata.py#L100-L103 where the regex patterns are defined at https://github.com/python/cpython/blob/43a6e4fa4934fcc0cbd83f7f3dc1b23a5f79f24b/Lib/importlib/metadata.py#L59-L63

So it's "expected" in Python 3.9 that it will return a re.Match :-(

Ewen

ewenmcneill · 2023-10-07T09:29:12Z

Equivalent code in Python 3.10 is completely rearranged, but can be found at:

patterns: https://github.com/python/cpython/blob/97ce15c5f8743fb8b8967c6a05d6aec9fef9cbc7/Lib/importlib/metadata/__init__.py#L142-L146

extras: https://github.com/python/cpython/blob/97ce15c5f8743fb8b8967c6a05d6aec9fef9cbc7/Lib/importlib/metadata/__init__.py#L185-L188

and that doesn't obviously look that different, so I'm puzzled how this is a Python 3.9 specific bug. Maybe only Python 3.9 is leaking the audio-yamaha-opx module into visibility?

Ewen

Coerce each entry to be a a string, so we can use to check requirements. See GlasgowEmbedded#422

whitequark · 2023-10-07T13:20:40Z

Maybe only Python 3.9 is leaking the audio-yamaha-opx module into visibility?

This module relies on an optional aiohttp package, so that's why it's only happening with that module / that version.

And just to be clear, the code is supposed to work both with and without aiohttp being installed, in the latter case transparently handling it. I will not accept a workaround that does not restore the full expected functionality, and this issue will remain open until I or someone else understands exactly what's going on.

txf- · 2023-10-07T13:35:51Z

I went back and omitted the aiohttp injection and that still worked, at least for the issue in the title.

whitequark · 2023-10-07T13:47:49Z

Nominating the issue for today's meeting; hopefully we'll be able to get a resolution there. (I still don't understand why it is happening, and this loading mechanism is a key part of Glasgow.)

ewenmcneill · 2023-10-07T20:15:52Z

FTR, I wondered if it was only on editable (-e) glasgow installs on Python 3.9, but no it appears to happen on both editable and non-editable installs. Either way (editable or not) the audio-yamaha-opx applet ends up being visible in the venv, and thus visible in dist-info entry-points.txt (which seems to be what is being parsed to find that http regex match.

AFAICT list(re.finditer(r'\w+', match.group('extras') or '')) is going to return a list of re.Match() objects, and that'll happen if there are any extras. The only thing I don't understand is how this apparently doesn't happen with Python 3.10?!

ETA: Python 3.12 re.findall() suggests re.finditer() will return re.Match objects, and re.findall() will return strings. Which seems to be fairly similar in Python 3.9 re.findall() and re.finditer(). But the code being run according to the debugger is re.finditer()...

Ewen

Python 3.9 pipx non-editable install, still fails with TypeError

ewen@parthenon:~$ cd /src
ewen@parthenon:/src$ rm -r "${PIPX_HOME}/venvs/glasgow" "${PIPX_BIN_DIR}/glasgow"
ewen@parthenon:/src$ which glasgow
ewen@parthenon:/src$ (cd glasgow && git status && git diff origin/main main && git log -1 --oneline)
On branch main
Your branch is up to date with 'origin/main'.

nothing to commit, working tree clean
f1e9185 (HEAD -> main, origin/main, origin/HEAD) manual: redirect from our domain to /latest/intro.html, not to the index.
ewen@parthenon:/src$ pipx3.9 install --spec 'glasgow/software[builtin-toolchain]' glasgow
  installed package glasgow 0.1.dev1874+gf1e9185, Python 3.9.5
  These binaries are now globally available
    - glasgow
done! ✨ 🌟 ✨
ewen@parthenon:/src$ which glasgow
/usr/local/share/pipx/bin/glasgow
ewen@parthenon:/src$ glasgow --version
Traceback (most recent call last):
  File "/usr/local/share/pipx/bin/glasgow", line 8, in <module>
    sys.exit(main())
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/cli.py", line 903, in main
    exit(loop.run_until_complete(_main()))
  File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/cli.py", line 486, in _main
    args = get_argparser().parse_args()
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/cli.py", line 279, in get_argparser
    add_applet_arg(p_run, mode="interact", required=True)
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/cli.py", line 124, in add_applet_arg
    for handle, metadata in GlasgowAppletMetadata.all().items():
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/support/plugin.py", line 87, in all
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/support/plugin.py", line 87, in <dictcomp>
    return {ep.name: cls(ep) for ep in _entry_points(group=cls.GROUP_NAME)}
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/support/plugin.py", line 97, in __init__
    self.requirements = _requirements_for_optional_dependencies(
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/glasgow/support/plugin.py", line 31, in _requirements_for_optional_dependencies
    if requirement.marker and requirement.marker.evaluate({"extra": dependency}):
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/packaging/markers.py", line 252, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/packaging/markers.py", line 157, in _evaluate_markers
    lhs_value, rhs_value = _normalize(lhs_value, rhs_value, key=environment_key)
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/packaging/markers.py", line 131, in _normalize
    return tuple(canonicalize_name(v) for v in values)
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/packaging/markers.py", line 131, in <genexpr>
    return tuple(canonicalize_name(v) for v in values)
  File "/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages/packaging/utils.py", line 47, in canonicalize_name
    value = _canonicalize_regex.sub("-", name).lower()
TypeError: expected string or bytes-like object
ewen@parthenon:/src$ 
ewen@parthenon:/src$ cd "${PIPX_HOME}/venvs/glasgow/lib/python3.9/site-packages/"
ewen@parthenon:/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages$ grep yamaha glasgow*dist-info/*entry*
audio-yamaha-opx = glasgow.applet.audio.yamaha_opx:AudioYamahaOPxApplet [http]
ewen@parthenon:/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages$ ls glasgow/applet/audio/yamaha_opx/
index.html  __init__.py  __pycache__
ewen@parthenon:/usr/local/share/pipx/venvs/glasgow/lib/python3.9/site-packages$

pdb on Python 3.9 importlib extras

ewen@parthenon:~$ . "${PIPX_HOME}/venvs/glasgow/bin/activate"
(glasgow) ewen@parthenon:~$ python3.9 -m pdb "${PIPX_BIN_DIR}/glasgow" --version
> /usr/local/share/pipx/venvs/glasgow/bin/glasgow(3)<module>()
-> import re
(Pdb) b /usr/lib/python3.9/importlib/metadata.py:93
Breakpoint 1 at /usr/lib/python3.9/importlib/metadata.py:93
(Pdb) c
> /usr/lib/python3.9/importlib/metadata.py(93)extras()
-> match = self.pattern.match(self.value)
(Pdb) p self.value
'glasgow.applet.interface.analyzer:AnalyzerApplet'
(Pdb) c
> /usr/lib/python3.9/importlib/metadata.py(93)extras()
-> match = self.pattern.match(self.value)
(Pdb) p self.value
'glasgow.applet.audio.dac:AudioDACApplet'
(Pdb) c
> /usr/lib/python3.9/importlib/metadata.py(93)extras()
-> match = self.pattern.match(self.value)
(Pdb) p self.value
'glasgow.applet.audio.yamaha_opx:AudioYamahaOPxApplet [http]'
(Pdb) n
> /usr/lib/python3.9/importlib/metadata.py(94)extras()
-> return list(re.finditer(r'\w+', match.group('extras') or ''))
(Pdb) p match
<re.Match object; span=(0, 59), match='glasgow.applet.audio.yamaha_opx:AudioYamahaOPxApp>
(Pdb) p match.group('extras')
'[http]'
(Pdb) p list(re.finditer(r'\w+', match.group('extras') or ''))
[<re.Match object; span=(1, 5), match='http'>]
(Pdb)

ewenmcneill · 2023-10-07T20:36:01Z

ETA: Python 3.12 re.findall() suggests re.finditer() will return re.Match objects, and re.findall() will return strings. Which seems to be fairly similar in Python 3.9 re.findall() and re.finditer(). But the code being run according to the debugger is re.finditer()...

Oh no.

ewen@parthenon:~$ grep -A 3 "def extras" /usr/lib/python3.9/importlib/metadata.py
    def extras(self):
        match = self.pattern.match(self.value)
        return list(re.finditer(r'\w+', match.group('extras') or ''))

ewen@parthenon:~$

which does not match the Python 3.9.latest I found in GitHub last night (in #422 (comment)), which is:

    @property
    def extras(self):
        match = self.pattern.match(self.value)
        return re.findall(r'\w+', match.group('extras') or '')

in https://github.com/python/cpython/blob/43a6e4fa4934fcc0cbd83f7f3dc1b23a5f79f24b/Lib/importlib/metadata.py#L100-L103

And the reason it doesn't match is python/cpython#91160, which backported some importlib_metadata fixes to Python 3.9..... which leads us to this exact problem reported in 2022: python/importlib_metadata#369, and this importlib_metadata fix that switched to re.findall() -- python/importlib_metadata@14cce75

Relevant Python upstream bug for the backports is https://bugs.python.org/issue47004. Looks like the relevant backport was merged around 2022-03-13. Looks like that backport is in Python 3.9.12 (released 2022-03-23), see:

https://github.com/python/cpython/blob/v3.9.12/Lib/importlib/metadata.py#L100-L103

So AFAICT this issue affects Python 3.9 <= 3.9.11 :-(

And TBH, it's possibly best fixed by hand patching python3.9/importlib/metadata.py.

ETA: Looks like it'll also affect Python 3.10 releases prior to March 2022.as well; fixed in 3.10 branch by backport python/cpython@d929aa7. Broken in Python 3.10.2:

https://github.com/python/cpython/blob/v3.10.2/Lib/importlib/metadata/__init__.py#L176-L179

and fixed in Python 3.10.3:

https://github.com/python/cpython/blob/v3.10.3/Lib/importlib/metadata/__init__.py#L185-L188

(Python 3.10.3 was released 2022-03-16).

Ewen

attie · 2023-10-07T23:32:26Z

Good/bad:

3.9.x
- 3.9.5 bad (see Ubuntu 20.04 LTS)
- 3.9.9 bad
- 3.9.10 bad
- 3.9.11 good
- 3.9.12 good
3.10.x
- 3.10.1 bad
- 3.10.2 bad
- 3.10.3 good

docker run --rm -it python:${version} bash

pip install 'git+https://github.com/GlasgowEmbedded/glasgow#subdirectory=software'
glasgow --version

Fixes GlasgowEmbedded#422.

whitequark · 2023-10-08T06:31:21Z

Reach heaven by violence. #436

Fixes #422.

ewenmcneill · 2023-10-08T07:56:17Z

Thank you for "fixing" this (monkeypatching FTW!); works for me on Ubuntu 20.04 LTS with Python 3.9.5 from Universe.

ewen@parthenon:/src$ grep VERSION= /etc/os-release; glasgow --version
VERSION="20.04.6 LTS (Focal Fossa)"
Glasgow version 0.1.dev1879+g42b7086 (Python 3.9.5)
ewen@parthenon:/src$ (cd glasgow && git log -1 --oneline)
42b7086 (HEAD -> main, origin/main, origin/HEAD) support.plugin: add workaround for defect in Python 3.9 <3.9.11.
ewen@parthenon:/src$

For anyone finding this in future, my Ubuntu 20.04 LTS install was done by:

Enabling "universe" repo if it is not already enabled
Installing python 3.9.5 from Ubuntu 20.04 universe, and an old packaged pipx:

sudo apt-get install python3.9 python3.9-venv python3.9-dev python3-pip pipx

Updating the pipx for a python3.9 option removal:

sudo vi /usr/lib/python3/dist-packages/pipx/Venv.py

and in _venv_metadata_for_package(), comment out the:

encoding="utf-8"

which was removed from Python3.9. (Or alternatively pick some other way to install a much more recent pipx, and have a nicer time as the main glasgow instructions are more likely to work for you.)

Using this pipx command instead of the one in the main instructions, due to (a) needing to override the Python version, and (b) the old pipx not supporting the all of the current features:

python3.9 /usr/bin/pipx install -e --verbose --spec 'glasgow/software[builtin-toolchain,http]' glasgow

(You may be able to leave ,http out; it pulls in the extra dependencies for the extra applet which was triggering this bug.)

All going well you should then be able to run glasgow --version and get output like the above.

Ewen

whitequark · 2023-10-08T22:57:33Z

Or alternatively pick some other way to install a much more recent pipx, and have a nicer time as the main glasgow instructions are more likely to work for you

I think you should be able to sudo pip install pipx or something like that?

ewenmcneill · 2023-10-09T00:06:52Z

I think you should be able to sudo pip install pipx or something like that?

Yes, the Python in Ubuntu 20.04 LTS (both 3.8, and universe 3.9.5) seems to be old enough not to complain about doing system-wide installs into the "managed Python".

This combination seems to work for me in an Ubuntu 20.04 LTS test docker run (although in the ubuntu:20.04 docker image you do have to do apt-get update && apt-get install sudo first).

sudo apt-get update
sudo apt-get install python3.9 python3-pip python3.9-venv git
sudo python3.9 -m pip install pipx
python3.9 /usr/local/bin/pipx ensurepath
git clone https://github.com/GlasgowEmbedded/glasgow
python3.9 /usr/local/bin/pipx install -e --verbose 'glasgow/software[builtin-toolchain,http]' 
glasgow --version

Note in particular that the pipx install syntax has changed from the "using old Ubuntu 20.40 packaged python-pipx" syntax because the old pipx doesn't work with the new command line syntax (cannot find package, removes venv), and the new pipx from PyPI doesn't understand the old --spec arguments. Apparently backwards compatibility is hard :-/

But as a general recommendation to Ubuntu 20.04 LTS users, using the pipx from PyPI is going to be a much nicer experience, as there seems to be a heap of changes in the least 3-4 years. (In particularly the old packaged python-pipx doesn't know about pipx reinstall PACKAGE either.)

Ewen

whitequark added the software Component: software label Oct 4, 2023

ewenmcneill added a commit to ewenmcneill/glasgow that referenced this issue Oct 7, 2023

Python 3.9 importlib.metadata extras is a re.Match

d51b001

Coerce each entry to be a a string, so we can use to check requirements. See GlasgowEmbedded#422

whitequark added the nominated Meta: nominated for next meeting label Oct 7, 2023

attie assigned whitequark Oct 7, 2023

attie removed the nominated Meta: nominated for next meeting label Oct 7, 2023

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

d62a67b

Fixes GlasgowEmbedded#422.

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

6f879f4

Fixes GlasgowEmbedded#422.

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

4f1388c

Fixes GlasgowEmbedded#422.

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

2f955d8

Fixes GlasgowEmbedded#422.

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

1051c3a

Fixes GlasgowEmbedded#422.

whitequark added a commit to whitequark/glasgow that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

73bf744

Fixes GlasgowEmbedded#422.

whitequark mentioned this issue Oct 8, 2023

Fix issue #422 #436

Merged

github-merge-queue bot pushed a commit that referenced this issue Oct 8, 2023

support.plugin: add workaround for defect in Python 3.9 <3.9.11.

42b7086

Fixes #422.

whitequark closed this as completed in #436 Oct 8, 2023

ewenmcneill mentioned this issue Oct 9, 2023

Argument 'bases' has incorrect type pydantic/pydantic#7689

Open

15 tasks

whitequark mentioned this issue Oct 21, 2023

Documentation #312

Open

35 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

running Glasgow exectutable produces "TypeError: expected string or bytes-like object" in packaging\utils.py #422

running Glasgow exectutable produces "TypeError: expected string or bytes-like object" in packaging\utils.py #422

txf- commented Oct 4, 2023 •

edited

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023 •

edited

ewenmcneill commented Oct 7, 2023 •

edited

txf- commented Oct 7, 2023 •

edited

ewenmcneill commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023

whitequark commented Oct 7, 2023 •

edited

txf- commented Oct 7, 2023 •

edited

whitequark commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023 •

edited

ewenmcneill commented Oct 7, 2023 •

edited

attie commented Oct 7, 2023 •

edited

whitequark commented Oct 8, 2023

ewenmcneill commented Oct 8, 2023

whitequark commented Oct 8, 2023

ewenmcneill commented Oct 9, 2023 •

edited

running Glasgow exectutable produces "TypeError: expected string or bytes-like object" in packaging\utils.py #422

running Glasgow exectutable produces "TypeError: expected string or bytes-like object" in packaging\utils.py #422

Comments

txf- commented Oct 4, 2023 • edited

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023

whitequark commented Oct 4, 2023

txf- commented Oct 4, 2023 • edited

ewenmcneill commented Oct 7, 2023 • edited

txf- commented Oct 7, 2023 • edited

ewenmcneill commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023

whitequark commented Oct 7, 2023 • edited

txf- commented Oct 7, 2023 • edited

whitequark commented Oct 7, 2023

ewenmcneill commented Oct 7, 2023 • edited

ewenmcneill commented Oct 7, 2023 • edited

attie commented Oct 7, 2023 • edited

whitequark commented Oct 8, 2023

ewenmcneill commented Oct 8, 2023

whitequark commented Oct 8, 2023

ewenmcneill commented Oct 9, 2023 • edited

txf- commented Oct 4, 2023 •

edited

txf- commented Oct 4, 2023 •

edited

ewenmcneill commented Oct 7, 2023 •

edited

txf- commented Oct 7, 2023 •

edited

whitequark commented Oct 7, 2023 •

edited

txf- commented Oct 7, 2023 •

edited

ewenmcneill commented Oct 7, 2023 •

edited

ewenmcneill commented Oct 7, 2023 •

edited

attie commented Oct 7, 2023 •

edited

ewenmcneill commented Oct 9, 2023 •

edited