Merge from 5.x: PR #17612

Fixes #17609
Fixes #17551

.github/workflows/installer-macos.yml

@@ -40,7 +40,6 @@ jobs:
           fi
           ${pythonLocation}/bin/python -m pip install -U pip setuptools wheel
           ${pythonLocation}/bin/python -m pip install -r req-build.txt -r req-extras.txt -r req-plugins.txt "${INSTALL_FLAGS[@]}" -e ${GITHUB_WORKSPACE}
-          ${pythonLocation}/bin/python -m pip uninstall -q -y spyder
       - name: Install Subrepos
         if: ${{github.event_name == 'pull_request'}}
         run: |

installers/macOS/Readme.md

@@ -14,60 +14,76 @@ Once you have the above requirements, you will create a virtual environment from
 In principle, it doesn't matter where your Python installation comes from except that it cannot come from Anaconda.
 I do not know exactly why an Anaconda installation does not work except that it has something to do with hardlinks.
 
-I recommend using [Homebrew](http://brew.sh/) to install pyenv and pyenv-virtualenv (if you plan to use these for virtual environment management).
-If you plan to use pyenv, you don't even need to install Python from Homebrew, since pyenv will install whatever python version you request.
-If you don't plan to use pyenv, then you will need to install Python from Homebrew or elsewhere.
+I recommend using [Homebrew](http://brew.sh/) to install `pyenv` and `pyenv-virtualenv` (if you plan to use these for virtual environment management).
+If you plan to use `pyenv`, you don't even need to install Python from Homebrew, since `pyenv` will install whatever python version you request.
+If you don't plan to use `pyenv`, then you will need to install Python from Homebrew or elsewhere.
 
 After installing Homebrew, run:
 
 ```
-$ brew install pyenv, pyenv-virtualenv, xz
+$ brew install pyenv, pyenv-virtualenv, xz, tcl-tk
 ```
 
 `xz` is a package that provides compression algorithms that Python should be built with to satisfy some packages, namely `pandas`.
 
-The Python frameworks must be copied to the stand-alone application, so if you use pyenv you must enable frameworks in any Python installation that you plan to use to build the Spyder app.
+The Python frameworks must be copied to the stand-alone application, so if you use `pyenv` you must enable frameworks in any Python installation that you plan to use to build the Spyder app.
+Additionally, the `tcl-tk` libraries must be referenced.
+The following commands install Python with these considerations.
 
 ```
-$ PYTHON_CONFIGURE_OPTS=--enable-framework pyenv install <python version>
+$ TKPREFIX=$(brew --prefix tcl-tk)
+$ export PYTHON_CONFIGURE_OPTS="--enable-framework --with-tcltk-includes=-I${TKPREFIX}/include --with-tcltk-libs='-L${TKPREFIX}/lib -ltcl8.6 -ltk8.6'"
+$ pyenv install <python version>
 ```
 
 ### Create Virtual Environment
 
-If you currently have any conda environment(s) activated, then deactivate them completely, i.e. you should not be in any conda environment, not even base.
-
 Create the virtual environment and populate it with the necessary package requirements.
-If you are using pyenv with pyenv-virtualenv, it will look like this:
+If you are using `pyenv` with `pyenv-virtualenv`, it will look like this:
 
 ```
 $ pyenv virtualenv <python version> spy-build
 $ pyenv activate spy-build
 ```
 
-If you are using venv, creating the environment will look like this:
+If you are using `venv`, creating the environment will look like this:
 
 ```
 $ python -m venv --clear --copies spy-build
 $ source spy-build/bin/activate
 ```
 
-Now change your working directory to `installers/macOS` directory of your local Spyder repo and install the necessary packages.
+It's also a good idea to update `pip` and `setuptools`
 
 ```
-(spy-build) $ cd <path>/<to>/spyder/installers/macOS
-(spy-build) $ pip install -r req-build.txt -r req-extras.txt -c req-const.txt -e ../../
-(spy-build) $ pip uninstall -q -y spyder
+$ (spy-build) $ python -m pip install -U pip setuptools wheel
 ```
 
-This will install the packages required for building Spyder and extra packages like matplotlib.
-Spyder is installed from this repository for the sole purpose of getting all of its dependencies.
-It must be uninstalled since py2app will use the repository when creating the application bundle.
+Now change your working directory to the `installers/macOS` directory of your local Spyder repo and install the necessary packages.
+
+```
+(spy-build) $ cd <path>/<to>/spyder/installers/macOS
+(spy-build) $ python -m pip install -r req-build.txt -r req-extras.txt -r req-plugins.txt -r req-scientific.txt -e ../../
+```
 
+This will install Spyder, the packages required for building Spyder, and extra packages like matplotlib.
 `req-build.txt` contains only those packages required to build the stand-alone application.
-`req-extras.txt` contains optional packages to include, if desired, for use in IPython consoles launched from the "Same as Spyder" environment.
-If you use external environments, such as conda, for your IPython consoles, you don't need `req-extras.txt`.
+`req-extras.txt` contains optional packages that can be used by the Python language server.
+`req-plugins.txt` contains optional third party plugins.
+`req-scientific.txt` contains optional packages to include for use in IPython consoles launched from the "Same as Spyder" environment.
+If you use external environments, such as conda, for your IPython consoles, you don't need `req-scientific.txt`.
 The build command also provides an option to exclude these packages, so you may install them and still build the application without them.
-`req-const.txt` contains package constraints that are known to cause problems with the build process if not satisfied.
+
+If your Spyder repo is checked out at a release commit or a commit that does not require synchronized commits for `python-lsp-server`, `qdarkstyle`, `qtconsole`, or `spyder-kernels`, then you should be able to proceed to building the application in the next section.
+However, if you need to synchronize the commits of these packages, then you must install them from Spyder's `external-deps` subrepo directory.
+
+```
+(spy-build) $ python -m pip install --no-deps -e ../../external-deps/qdarkstyle
+(spy-build) $ python -m pip install --no-deps -e ../../external-deps/qtconsole
+(spy-build) $ python -m pip install --no-deps -e ../../external-deps/spyder-kernels
+(spy-build) $ export SETUPTOOLS_SCM_PRETEND_VERSION=`../../python pylsp_utils.py`
+(spy-build) $ python -m pip install --no-deps -e ../../external-deps/python-lsp-server
+```
 
 ### Create the Standalone Application
 
@@ -77,7 +93,7 @@ To create the standalone application and package it in a dmg disk image run:
 (spy-build) $ python setup.py --dmg
 ```
 
-Further usage documentation can be accessed via
+Documentation on the various build options can be accessed via
 
 ```
 (spy-build) $ python setup.py -h

installers/macOS/packages.py

@@ -20,10 +20,10 @@
     ImportError: The 'more_itertools' package is required; normally this is
     bundled with this package so if you get this warning, consult the
     packager of your distribution.
-pylsp_black :
-    Mandatory: python-pyls-black >=1.0.0 : None (NOK)
 pyls_spyder :
     Mandatory: pyls_spyder >=0.1.1 : None (NOK)
+pylsp_black :
+    Mandatory: python-pyls-black >=1.0.0 : None (NOK)
 setuptools :
     Mandatory: setuptools >=49.6.0 : None (NOK)
 spyder :
@@ -39,8 +39,8 @@
 PACKAGES = [
     'humanfriendly',
     'pkg_resources',
-    'pylsp_black',
     'pyls_spyder',
+    'pylsp_black',
     'setuptools',
     'spyder',
     'spyder_kernels',

installers/macOS/req-build.txt

@@ -1,4 +1,3 @@
 # For building standalone Mac app
-py2app>=0.27
+py2app>=0.28
 dmgbuild>=1.4.2
-setuptools<61.0.0  # remove when there is a better fix #17547

installers/macOS/setup.py

@@ -11,95 +11,36 @@
 $ python setup.py
 """
 
-import os
 import sys
 import shutil
 from logging import getLogger, StreamHandler, Formatter
 from pathlib import Path
 from setuptools import setup
 
+from spyder import __version__ as SPYVER
+from spyder.config.base import MAC_APP_NAME
+
 # Setup logger
 fmt = Formatter('%(asctime)s [%(levelname)s] [%(name)s] -> %(message)s')
 h = StreamHandler()
 h.setFormatter(fmt)
-logger = getLogger('spyder-macOS')
+logger = getLogger('macOS-installer')
 logger.addHandler(h)
 logger.setLevel('INFO')
 
 # Define paths
 THISDIR = Path(__file__).resolve().parent
 SPYREPO = (THISDIR / '..' / '..').resolve()
 ICONFILE = SPYREPO / 'img_src' / 'spyder.icns'
-SPYLINK = THISDIR / 'spyder'
+APPSCRIPT = SPYREPO / 'scripts' / 'spyder'
 
-sys.path.append(SPYREPO.as_posix())
-
-from spyder import __version__ as SPYVER
-from spyder.config.base import MAC_APP_NAME
+APP_BASE_NAME = MAC_APP_NAME[:-4]
 
 # Python version
 PYVER = [sys.version_info.major, sys.version_info.minor,
          sys.version_info.micro]
 
 
-def fix_zip_entry_points(zfile):
-    """
-    Fix zip archive so that pkg_resources will find entry points.
-    Remove if a better solution emerges.
-
-    Parameters
-    ----------
-    zfile : pathlib.Path
-        Path to zip archive.
-    """
-    import os
-    from zipfile import ZipFile
-
-    logger.info('Converting zip file...')
-
-    file = zfile.parent / 'temp'
-    ZipFile(zfile).extractall(file)
-    os.remove(zfile)
-    file.replace(zfile)
-
-
-def patch_py2app():
-    """
-    Patch py2app PyQt recipe and site.py for version 0.27.
-    Remove after version 0.28 is available.
-    """
-    from importlib.util import find_spec
-    from importlib.metadata import version
-    from packaging.version import parse
-
-    logger.info('Patching py2app...')
-
-    py2app_ver = version('py2app')
-    if parse(py2app_ver) > parse('0.27'):
-        raise DeprecationWarning(f'py2app version {py2app_ver} > 0.27; '
-                                 'stop using patch_py2app.')
-
-    root = Path(find_spec('py2app').origin).parent
-
-    # Patch site.py
-    site_file = root / 'apptemplate' / 'lib' / 'site.py'
-    append_text = ("builtins.quit = "
-                   "_sitebuiltins.Quitter('quit', 'Ctrl-D (i.e. EOF)')\n"
-                   "builtins.exit = "
-                   "_sitebuiltins.Quitter('exit', 'Ctrl-D (i.e. EOF)')\n")
-    text = site_file.read_text()
-    if append_text not in text:
-        site_file.write_text(text + append_text)
-
-    # Patch qt5.py
-    qt5_file = root / 'recipes' / 'qt5.py'
-    search_text = "if qtdir != os.path.dirname(PyQt5.__file__):"
-    replace_text = "if os.path.dirname(PyQt5.__file__) not in qtdir:"
-    text = qt5_file.read_text()
-    if replace_text not in text:
-        qt5_file.write_text(text.replace(search_text, replace_text))
-
-
 def make_app_bundle(dist_dir, make_lite=False):
     """
     Make macOS application bundle.
@@ -144,28 +85,9 @@ def make_app_bundle(dist_dir, make_lite=False):
         }
     }
 
-    # Copy main application script
-    app_script_name = MAC_APP_NAME.replace('.app', '.py')
-    app_script_path = SPYREPO / 'scripts' / app_script_name
-    shutil.copy2(SPYREPO / 'scripts' / 'spyder', app_script_path)
-
     # Build the application
-    try:
-        patch_py2app()
-        os.symlink(SPYREPO / 'spyder', SPYLINK)
-        setup(app=[app_script_path.as_posix()], options={'py2app': OPTIONS})
-        fix_zip_entry_points(
-            dist_dir / MAC_APP_NAME / 'Contents' / 'Resources' / 'lib' /
-            'python{}{}.zip'.format(*PYVER[:2]))
-    finally:
-        os.remove(app_script_path)
-        os.remove(SPYLINK)
-
-    # Copy Spyder egg-info
-    egg = SPYREPO / 'spyder.egg-info'
-    dest = (dist_dir / MAC_APP_NAME / 'Contents' / 'Resources' / 'lib' /
-            'python{}.{}'.format(*PYVER) / 'spyder.egg-info')
-    shutil.copytree(egg, dest)
+    setup(name=APP_BASE_NAME, app=[APPSCRIPT.as_posix()],
+          options={'py2app': OPTIONS})
 
     return
 
@@ -188,7 +110,7 @@ def make_disk_image(dist_dir, make_lite=False):
     from dmgbuild import build_dmg
     from dmgbuild.core import DMGError
 
-    volume_name = '{}-{} Py-{}.{}.{}'.format(MAC_APP_NAME[:-4], SPYVER, *PYVER)
+    volume_name = '{}-{} Py-{}.{}.{}'.format(APP_BASE_NAME, SPYVER, *PYVER)
     dmg_name = 'Spyder'
     if make_lite:
         volume_name += ' Lite'