FileExistsError macos with playwright webkit driver locally installed #8363

timrichardson · 2024-03-18T10:58:18Z

Description of the issue

On macos (intel, Ventura) when building a project, I get a FileExistsError
The most complex thing about this project is that has a locally installed playwright browser driver.

pyinstaller is 6.5.0
python is 3.11.8

The application runs correctly when run as a python script.

Context information (for bug reports)

Output of pyinstaller --version: (paste here)
6.5.0
Version of Python:
3.11.8
Platform:
os x. The same project builds fine on Windows, also with webkit driver.
How you installed Python:
python is a venv based on the official python installer.
Did you also try this on another platform? Does it work there?
Yes and yes.
try the latest development version, using the following command:

pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

The same problem.
--onedir produces the same result

(.venv) tim@tims-iMac-Pro dearAuth % PLAYWRIGHT_BROWSERS_PATH=0 pyinstaller dear_auth.py                       
102 INFO: PyInstaller: 6.5.0, contrib hooks: 2024.3
102 INFO: Python: 3.11.8
114 INFO: Platform: macOS-13.6.5-x86_64-i386-64bit
114 INFO: wrote /Users/tim/PycharmProjects/dearAuth/dear_auth.spec
118 INFO: Extending PYTHONPATH with paths
['/Users/tim/PycharmProjects/dearAuth']
214 INFO: checking Analysis
577 INFO: checking PYZ
581 INFO: EXE target arch: x86_64
581 INFO: Code signing identity: None
584 INFO: checking PKG
584 INFO: Bootloader /Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/bootloader/Darwin-64bit/run
584 INFO: checking EXE
658 INFO: checking COLLECT
658 INFO: Building COLLECT because COLLECT-00.toc is non existent
WARNING: The output directory "/Users/tim/PycharmProjects/dearAuth/dist/dear_auth" and ALL ITS CONTENTS will be REMOVED! Continue? (y/N)y
On your own risk, you can use the option `--noconfirm` to get rid of this question.
3368 INFO: Removing dir /Users/tim/PycharmProjects/dearAuth/dist/dear_auth
5222 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "/Users/tim/PycharmProjects/dearAuth/.venv/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 228, in _console_script_run
    run()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 212, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 69, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1139, in main
    build(specfile, distpath, workpath, clean_build)
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1079, in build
    exec(code, spec_namespace)
  File "/Users/tim/PycharmProjects/dearAuth/dear_auth.spec", line 35, in <module>
    coll = COLLECT(
           ^^^^^^^^
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/api.py", line 1092, in __init__
    self.__postinit__()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/api.py", line 1150, in assemble
    os.symlink(src_name, dest_path)  # Create link at dest_path, pointing at (relative) src_name
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: 'Versions/Current/Helpers' -> '/Users/tim/PycharmProjects/dearAuth/dist/dear_auth/_internal/playwright/driver/package/.local-browsers/webkit-1983/JavaScriptCore.framework/Helpers'
(.venv) tim@tims-iMac-Pro dearAuth %

follow all the instructions in our "If Things Go Wrong" Guide
(https://github.com/pyinstaller/pyinstaller/wiki/If-Things-Go-Wrong) and

Make sure everything is packaged correctly

start with clean installation
use the latest development version
Run your frozen program from a command window (shell) — instead of double-clicking on it
Package your program in --onedir mode
Package without UPX, say: use the option --noupx or set upx=False in your .spec-file
Repackage you application in verbose/debug mode. For this, pass the option --debug to pyi-makespec or pyinstaller or use EXE(..., debug=1, ...) in your .spec file.

A minimal example program which shows the error

Note: first you need to

pip install playwright

and from directory of script:

PLAYWRIGHT_BROWSERS_PATH=0 playwright install webkit

import json
import os
import time
from typing import Dict, List, Tuple, Any


from playwright.sync_api import sync_playwright, Page, BrowserContext, Browser
from tkinter import messagebox


# Set the PLAYWRIGHT_BROWSERS_PATH environment variable
os.environ['PLAYWRIGHT_BROWSERS_PATH'] = '0'


def show_browser_for_login(playwright_instance, org_id) -> Tuple[Page, BrowserContext, Browser]:
   """ called by fetch_cookied_from_dear"""
   # browser = playwright_instance.chromium.launch(headless=False)
   browser = playwright_instance.webkit.launch(headless=False)
   # browser = playwright_instance.firefox.launch(headless=False)




   context = browser.new_context()
   saved_cookies = None
   if saved_cookies:
       context.add_cookies(saved_cookies)
   page = context.new_page()
   login_url = f"""https://inventory.dearsystems.com/Home/Organisation?ID={org_id}"""  # login for TOQ
   page.goto(login_url)
   return page, context, browser

Stacktrace / full error message

(.venv) tim@tims-iMac-Pro dearAuth % PLAYWRIGHT_BROWSERS_PATH=0 pyinstaller minimal_reproducer.py --onedir
106 INFO: PyInstaller: 6.5.0, contrib hooks: 2024.3
106 INFO: Python: 3.11.8
114 INFO: Platform: macOS-13.6.5-x86_64-i386-64bit
115 INFO: wrote /Users/tim/PycharmProjects/dearAuth/minimal_reproducer.spec
118 INFO: Extending PYTHONPATH with paths
['/Users/tim/PycharmProjects/dearAuth']
185 INFO: checking Analysis
344 INFO: Building because /Users/tim/PycharmProjects/dearAuth/minimal_reproducer.py changed
344 INFO: Initializing module dependency graph...
345 INFO: Caching module graph hooks...
351 INFO: Analyzing base_library.zip ...
715 INFO: Loading module hook 'hook-encodings.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
957 INFO: Loading module hook 'hook-heapq.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
1218 INFO: Loading module hook 'hook-pickle.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
2562 INFO: Caching module dependency graph...
2603 INFO: Running Analysis Analysis-00.toc
2603 INFO: Looking for Python shared library...
2606 INFO: Using Python shared library: /Library/Frameworks/Python.framework/Versions/3.11/Python
2606 INFO: Analyzing /Users/tim/PycharmProjects/dearAuth/minimal_reproducer.py
2625 INFO: Loading module hook 'hook-playwright.sync_api.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/playwright/_impl/__pyinstaller'...
4572 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
4690 INFO: Loading module hook 'hook-xml.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
5104 INFO: Loading module hook 'hook-platform.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
5361 INFO: Loading module hook 'hook-_tkinter.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
5369 INFO: Processing module hooks...
5370 INFO: Loading module hook 'hook-_tkinter.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
5372 INFO: checking Tree
5375 INFO: checking Tree
5387 INFO: checking Tree
5612 INFO: Performing binary vs. data reclassification (19267 entries)
6638 INFO: Looking for ctypes DLLs
6646 INFO: Analyzing run-time hooks ...
6647 INFO: Including run-time hook '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_inspect.py'
6648 INFO: Including run-time hook '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
6649 INFO: Including run-time hook '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgutil.py'
6651 INFO: Processing pre-find module path hook _pyi_rth_utils from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks/pre_find_module_path/hook-_pyi_rth_utils.py'.
6652 INFO: Loading module hook 'hook-_pyi_rth_utils.py' from '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks'...
6652 INFO: Including run-time hook '/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
6921 INFO: Looking for dynamic libraries
8088 INFO: Warnings written to /Users/tim/PycharmProjects/dearAuth/build/minimal_reproducer/warn-minimal_reproducer.txt
8100 INFO: Graph cross-reference written to /Users/tim/PycharmProjects/dearAuth/build/minimal_reproducer/xref-minimal_reproducer.html
8574 INFO: checking PYZ
8577 INFO: EXE target arch: x86_64
8577 INFO: Code signing identity: None
8578 INFO: checking PKG
8579 INFO: Building because /Users/tim/PycharmProjects/dearAuth/minimal_reproducer.py changed
8579 INFO: Building PKG (CArchive) minimal_reproducer.pkg
8586 INFO: Building PKG (CArchive) minimal_reproducer.pkg completed successfully.
8587 INFO: Bootloader /Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/bootloader/Darwin-64bit/run
8587 INFO: checking EXE
8587 INFO: Rebuilding EXE-00.toc because pkg is more recent
8587 INFO: Building EXE from EXE-00.toc
8587 INFO: Copying bootloader EXE to /Users/tim/PycharmProjects/dearAuth/build/minimal_reproducer/minimal_reproducer
8588 INFO: Converting EXE to target arch (x86_64)
8606 INFO: Removing signature(s) from EXE
8636 INFO: Appending PKG archive to EXE
8637 INFO: Fixing EXE headers for code signing
8649 INFO: Re-signing the EXE
8678 INFO: Building EXE from EXE-00.toc completed successfully.
8801 INFO: checking COLLECT
8801 INFO: Building COLLECT because COLLECT-00.toc is non existent
WARNING: The output directory "/Users/tim/PycharmProjects/dearAuth/dist/minimal_reproducer" and ALL ITS CONTENTS will be REMOVED! Continue? (y/N)y
On your own risk, you can use the option `--noconfirm` to get rid of this question.
15475 INFO: Removing dir /Users/tim/PycharmProjects/dearAuth/dist/minimal_reproducer
17522 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "/Users/tim/PycharmProjects/dearAuth/.venv/bin/pyinstaller", line 8, in <module>
    sys.exit(_console_script_run())
             ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 228, in _console_script_run
    run()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 212, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/__main__.py", line 69, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1139, in main
    build(specfile, distpath, workpath, clean_build)
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/build_main.py", line 1079, in build
    exec(code, spec_namespace)
  File "/Users/tim/PycharmProjects/dearAuth/minimal_reproducer.spec", line 35, in <module>
    coll = COLLECT(
           ^^^^^^^^
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/api.py", line 1092, in __init__
    self.__postinit__()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/datastruct.py", line 184, in __postinit__
    self.assemble()
  File "/Users/tim/PycharmProjects/dearAuth/.venv/lib/python3.11/site-packages/PyInstaller/building/api.py", line 1150, in assemble
    os.symlink(src_name, dest_path)  # Create link at dest_path, pointing at (relative) src_name
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: 'Versions/Current/Helpers' -> '/Users/tim/PycharmProjects/dearAuth/dist/minimal_reproducer/_internal/playwright/driver/package/.local-browsers/webkit-1983/JavaScriptCore.framework/Helpers'
(.venv) tim@tims-iMac-Pro dearAuth %

Please also see https://github.com/pyinstaller/pyinstaller/wiki/How-to-Report-Bugs
for more about what would use to solve the issue.

The text was updated successfully, but these errors were encountered:

rokm · 2024-03-18T11:06:38Z

Yeah, collection of stuff that playwright installs into its package directory is broken on macOS, and with PyInstaller 6.x, this is even more pronounced than with 5.x series (see also #7969).

Using last PyInstaller 5.x release (5.13.2) might alleviate the problem, at the cost of all those symlinks being duplicated and/or missing.

timrichardson · 2024-03-18T12:24:47Z

then the browser just doesn't work in the compiled app. It does not launch

rokm · 2024-03-18T12:25:56Z

then the browser just doesn't work in the compiled app. It does not launch

Then in contrast to #7969, this never really worked.

timrichardson · 2024-03-18T12:49:57Z

Using 5.13.2 I can get it working with the chromium and firefox driver but not webkit (which I was using). However, I changed one other thing apart from downgrading pyinstaller, I also removed setting the env var PLAYWRIGHT_BROWSERS_PATH=0 from inside my python script.
So I am very relieved to have a binary that works at least on my intel macos vm. That is enough for the moment.
While I tried every driver with 6.5.0, I always was setting PLAYWRIGHT_BROWSERS_PATH=0 from inside the script, perhaps that was significant.

rokm · 2024-03-18T12:57:00Z

PLAYWRIGHT_BROWSERS_PATH=0 causes browser to be downloaded in the package directory, right? So you might also be able to use v6 without PLAYWRIGHT_BROWSERS_PATH=0, but you need a clean installation of playwright (i.e., one that has never downloaded a browser in its local package directory). Otherwise, whatever browsers are downloaded there end up being collected (which causes errors in v6, and unnecessarily bloats up the build in v5).

timrichardson · 2024-03-18T12:59:12Z

when installing the drivers, we have to use PLAYWRIGHT_BROWSERS_PATH=0 because otherwise there is no binary driver for the browser in the package, and nothing will work.
IN order for the package runtime, I though I also had to provide the same var but now I am not so sure.

timrichardson added the triage Please triage and relabel this issue label Mar 18, 2024

rokm added bug and removed triage Please triage and relabel this issue labels Mar 18, 2024

gruve-p mentioned this issue May 3, 2024

build: update bundled python version spesmilo/electrum#9013

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FileExistsError macos with playwright webkit driver locally installed #8363

FileExistsError macos with playwright webkit driver locally installed #8363

timrichardson commented Mar 18, 2024

rokm commented Mar 18, 2024

timrichardson commented Mar 18, 2024

rokm commented Mar 18, 2024

timrichardson commented Mar 18, 2024

rokm commented Mar 18, 2024 •

edited

timrichardson commented Mar 18, 2024 •

edited

FileExistsError macos with playwright webkit driver locally installed #8363

FileExistsError macos with playwright webkit driver locally installed #8363

Comments

timrichardson commented Mar 18, 2024

Description of the issue

Context information (for bug reports)

Make sure everything is packaged correctly

A minimal example program which shows the error

Stacktrace / full error message

rokm commented Mar 18, 2024

timrichardson commented Mar 18, 2024

rokm commented Mar 18, 2024

timrichardson commented Mar 18, 2024

rokm commented Mar 18, 2024 • edited

timrichardson commented Mar 18, 2024 • edited

rokm commented Mar 18, 2024 •

edited

timrichardson commented Mar 18, 2024 •

edited