os.environ didn't return unicode after compiled
#7989
Comments
T-256
changed the title
os.environ don't return unicode after compiledos.environ didn't return unicode after compiled
|
It seems to work as expected on my test Windows 10 system, with python 3.12.0 (32-bit installer from Python.org) and PyInstaller 5.13.2 (since the end of your build log suggests you are using PyInstaller < 6.0.0). I copied your username
Can you run my version of the program program.pyimport os
import sys
import pathlib
appdata = os.environ["APPDATA"]
print(f"appdata = {appdata!r}")
print(f"appdata len = {len(appdata)!r}")
print(f"appdata encoded = {appdata.encode()!r}")
print(f"appdata encoded len = {len(appdata.encode())!r}")
print("")
print(f"sys.flags = {sys.flags!r}")
print(f"sys.getdefaultencoding() = {sys.getdefaultencoding()!r}")
print(f"sys.getfilesystemencoding() = {sys.getfilesystemencoding()!r}")
print(f"sys.getfilesystemencodeerrors() = {sys.getfilesystemencodeerrors()!r}")
print("")
appdata_path = pathlib.Path(appdata).resolve()
print(f"contents of {appdata_path!r}")
for entry in appdata_path.iterdir():
print(f" {entry!r}")so we can see the detailed information? |
|
Also, does building with PyInstaller v6 make any difference? |
|
I'm using win10-1507, official cpython 3.12 32bit |
|
with force # -*- mode: python ; coding: utf-8 -*-
a = Analysis(
['app.py'],
pathex=[],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=[],
noarchive=False,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[('X utf8_mode=1', None, 'OPTION')],
exclude_binaries=True,
name='app',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True,
disable_windowed_traceback=False,
argv_emulation=False,
target_arch=None,
codesign_identity=None,
entitlements_file=None,
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=True,
upx_exclude=[],
name='app',
)it seems still remains |
Yeah, I just noticed the examples in documentation are wrong - it should be just or even just Does that make a difference? |
|
|
Can you upload your build somewhere so I can try running it in my test environment? This way, we can rule out the differences in build - which will likely means that the different behavior is caused by some local system setting. |
|
Also, do you explicitly enable utf8 mode in your python? Do you have |
I think it's because of that OS. I tested exact built app on win11-22h2 and works with these flags (no new change):
No. pure install on VM. Env vars: |
Indeed, it works for me as well. Up-to-date win10 (19045.3516). What's the version of the system where it doesn't work? |
So, if issue is with OS, the main quesion is why python interpreter works on it?
Windows 10 1507 (Build 10240) - 64-bit |
No idea, really. I'll have to test with 1507, and if I can reproduce the issue, see what is actually going on. |
|
FWIW, I tried interact directly with kernel32:
I don't understand why there are visual characters difference, but good point is that working. |
|
I also tried to decode Perhaps somewhere in compiled app using one of these encodings. UPDATEI found that shell uses Where compiled app decided to use I also tried |
|
From what I can tell, the behavior difference between python and frozen application comes from the way the bootloader is compiled (which makes sense, because the bootloader itself does not touch The long story is, I set up a 1507 machine for test, and could not install VS2022 in it ("Unsupported Windows version"), so I decided to go with msys2; there, I had to decide between MINGW64 and UCRT64 toolchain, and ended up testing with both. If I build bootloader with MINGW64 (which supposedly uses the legacy msvcrt runtime), I can reproduce the issue. If I build bootloader with UCRT64 (which uses the new ucrt runtime), the encoding issue is gone. And it's likely that some Windows upgrade later on fixed the msvcrt runtime to behave more like ucrt, or something along those lines, so you get different behavior only on old Windows 10 version(s). So I'll need to check which runtime the bootloader ends up using when built with MSVC (and what options we should set in wscript to use ucrt, if possible). |
|
https://www.msys2.org/docs/environments:
I think the reason is |
|
Actually, it might be a matter of |
|
Yeah, this is going to be a pain one way or another... When building with MSVC, we are not specifying either And mixing both modes is discouraged, as it can lead to subtle issues, such as the one we've seen here (where either the state of the private copy of CRT used by bootloader is out of sync with the global/shared state used by python shared library, or there is some other sort of incompatibility between the bootloader-private copy of CRT and the global/shared one). Now of course, the correct way to fix this would be to build with FWIW, I am quite fine with users having to install UCRT (which is part of the OS in Windows 10 and 11) on older systems. When building the frozen application, whether The VC runtime, however, is not a part of the OS even on 10, and judging by the issues with We can go half-way, though, by doing "semi-static" linking: link Here is a proof-of-concept branch, if you want to test it. I've included rebuilt bootloaders for easier testing: You can install it directly with followed by or |
rokm
added
bug
and removed
state:need info
My app depends on
APPDATAenv var, and it crashes on system that have unicode username:Note that it passes "question mark" instead of unicode characters.
The text was updated successfully, but these errors were encountered: