[6.0, MacOS] Chromium.app/Contents/MacOS/Chromium: bundle format unrecognized, invalid, or unsuitable

[kwist-sgr]

Description of the issue

Context information (for bug reports)

• Output of pyinstaller --version: 6.0.0 (v5.13.2 works)

• Version of Python: v3.11.5

• Platform: macOS-12.7-x86_64-i386-64bit

• How you installed Python: pyenv

• Did you also try this on another platform? Does it work there?: Yep it works on Liniux/Windows platform.

• try the latest development version, using the following command: Likewise - it doesn't work

• follow all the instructions in our "If Things Go Wrong" Guide
  (https://github.com/pyinstaller/pyinstaller/wiki/If-Things-Go-Wrong) and

Make sure everything is packaged correctly

• start with clean installation
• use the latest development version
• Run your frozen program from a command window (shell) — instead of double-clicking on it
• Package your program in --onedir mode
• Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
• Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

# workon into virtualenv
$ pip install playwright
$ PLAYWRIGHT_BROWSERS_PATH=0 python -m playwright install chromium

#!/usr/bin/env python
from playwright.sync_api import sync_playwright


def main():
    with sync_playwright() as p:
        browser = p.chromium.launch()
        page = browser.new_page()
        page.goto("http://playwright.dev")
        print(page.title())
        browser.close()


if __name__ == '__main__':
    main()

Stacktrace / full error message

$ pyinstaller --clean --onefile --noconfirm --name demo demo.py
78 INFO: PyInstaller: 6.0.0
78 INFO: Python: 3.11.5
86 INFO: Platform: macOS-14.0-arm64-arm-64bit
87 INFO: wrote /Users/kwist/workspace/temp/pyinstaller-bug/demo.spec
89 INFO: Removing temporary files and cleaning cache in /Users/kwist/Library/Application Support/pyinstaller
90 INFO: Extending PYTHONPATH with paths
['/Users/kwist/workspace/temp/pyinstaller-bug']
143 INFO: checking Analysis
143 INFO: Building Analysis because Analysis-00.toc is non existent
143 INFO: Initializing module dependency graph...
144 INFO: Caching module graph hooks...
148 INFO: Analyzing base_library.zip ...
582 INFO: Loading module hook 'hook-heapq.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
626 INFO: Loading module hook 'hook-encodings.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
1506 INFO: Loading module hook 'hook-pickle.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
2175 INFO: Caching module dependency graph...
2211 INFO: Running Analysis Analysis-00.toc
2211 INFO: Looking for Python shared library...
2213 INFO: Using Python shared library: /Users/kwist/.pyenv/versions/3.11.5/lib/libpython3.11.dylib
2213 INFO: Analyzing /Users/kwist/workspace/temp/pyinstaller-bug/demo.py
2215 INFO: Loading module hook 'hook-playwright.sync_api.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/playwright/_impl/__pyinstaller'...
2561 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
2659 INFO: Loading module hook 'hook-xml.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
3105 INFO: Loading module hook 'hook-platform.py' from '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks'...
3254 INFO: Processing module hooks...
3589 INFO: Looking for ctypes DLLs
3595 INFO: Analyzing run-time hooks ...
3597 INFO: Including run-time hook '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
3597 INFO: Including run-time hook '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
3598 INFO: Including run-time hook '/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
3610 INFO: Looking for dynamic libraries
3797 INFO: Warnings written to /Users/kwist/workspace/temp/pyinstaller-bug/build/demo/warn-demo.txt
3805 INFO: Graph cross-reference written to /Users/kwist/workspace/temp/pyinstaller-bug/build/demo/xref-demo.html
3821 INFO: checking PYZ
3821 INFO: Building PYZ because PYZ-00.toc is non existent
3821 INFO: Building PYZ (ZlibArchive) /Users/kwist/workspace/temp/pyinstaller-bug/build/demo/PYZ-00.pyz
4047 INFO: Building PYZ (ZlibArchive) /Users/kwist/workspace/temp/pyinstaller-bug/build/demo/PYZ-00.pyz completed successfully.
4049 INFO: EXE target arch: arm64
4049 INFO: Code signing identity: None
4055 INFO: checking PKG
4055 INFO: Building PKG because PKG-00.toc is non existent
4055 INFO: Building PKG (CArchive) demo.pkg
Traceback (most recent call last):
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/utils.py", line 282, in process_collected_binary
    osxutils.sign_binary(cached_name, codesign_identity, entitlements_file)
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/utils/osx.py", line 386, in sign_binary
    raise SystemError(f"codesign command ({cmd_args}) failed with error code {p.returncode}!\noutput: {p.stdout}")
SystemError: codesign command (['codesign', '-s', '-', '--force', '--all-architectures', '--timestamp', '/Users/kwist/Library/Application Support/pyinstaller/bincache00py31164bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1080/chrome-mac/Chromium.app/Contents/MacOS/Chromium']) failed with error code 1!
output: /Users/kwist/Library/Application Support/pyinstaller/bincache00py31164bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1080/chrome-mac/Chromium.app/Contents/MacOS/Chromium: replacing existing signature
/Users/kwist/Library/Application Support/pyinstaller/bincache00py31164bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1080/chrome-mac/Chromium.app/Contents/MacOS/Chromium: bundle format unrecognized, invalid, or unsuitable
In subcomponent: /Users/kwist/Library/Application Support/pyinstaller/bincache00py31164bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1080/chrome-mac/Chromium.app/Contents/Frameworks/Chromium Framework.framework


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/kwist/.pyenv/versions/pyinst/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/__main__.py", line 205, in _console_script_run
    run()
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/__main__.py", line 189, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/__main__.py", line 61, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1033, in main
    build(specfile, distpath, workpath, clean_build)
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 973, in build
    exec(code, spec_namespace)
  File "/Users/kwist/workspace/temp/pyinstaller-bug/demo.spec", line 18, in <module>
    exe = EXE(
          ^^^^
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/api.py", line 606, in __init__
    self.pkg = PKG(
               ^^^^
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/api.py", line 238, in __init__
    self.__postinit__()
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/api.py", line 289, in assemble
    src_name = process_collected_binary(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/kwist/.pyenv/versions/3.11.5/envs/pyinst/lib/python3.11/site-packages/PyInstaller/building/utils.py", line 301, in process_collected_binary
    raise SystemError(f"Failed to process binary {cached_name!r}!") from e
SystemError: Failed to process binary '/Users/kwist/Library/Application Support/pyinstaller/bincache00py31164bit/arm64/adhoc/no-entitlements/playwright/driver/package/.local-browsers/chromium-1080/chrome-mac/Chromium.app/Contents/MacOS/Chromium'!

Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs
for more about what would use to solve the issue.

[rokm]

Huh... so playwright (and their PyInstaller hooks) expect us to be able to properly collect whole Chrome .app bundle that they stuff in their package? I really wish I could go back to the time when I possessed that sort of naivety...

This might have worked in v5 (if you are willing to ignore part where the 300 MB Chromium Framework shared lib is collected in three copies due to the way .framework bundles are symlinked), but only because the whole thing was smuggled in as data files. And it is now running afoul of binary processing that's triggered by automatic binary-vs-data reclassification.

I suppose we will have to extend the .framework bundle handling code to .app bundles as well.

And the binary processing code (signing in particular) will need to be revised to explicitly handling .framework and .app bundles; i.e., copy all their contents, and only then attempt a (re)sign - I suspect that is the case why this is blowing up now; we copy only the binary in question to bincache for path rewriting and (re)signing, and it then fails because for example Info.plist is missing.

I'm marking this as a bug, although the scope probably makes it more of a feature.

[benjie-git]

+1
I also need to include a mac app bundle embedded into my pyinstaller mac app. I'll have to go back to v5 for now.
(Not a full Chromimum! But an accessory app.)

[rokm]

@benjie-git How do you include that .app bundle in the PyInstaller app? It is presumably smuggled in as data (otherwise I think v5 would also raise an error), but do you use --add-data (or specify datas in the hook), or collect_data_files in a hook?

[benjie-git]

Thanks! In my spec file, I use a data parameter in my Analysis() call that adds a directory containing my sub-app to the build, like this:

datas=[('containerDir', '.')],

During my build process, conainerDir contains my SubApp.app bundle.

I am then even able to run code signing on the subapp, and then on the full app, and everything works smoothly for distribution as a trusted app on MacOS.

FYI, my subapp is also built using PyInstaller, earlier in my build process.