New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve packaging #3459 #3761

Open

pombredanne wants to merge 29 commits into develop from 3459-improve-packaging

Member

pombredanne commented May 1, 2024 •

edited

This PR improves packaging of ScanCode by adopting flot as a build tool
With this we have an overall simpler and more explicit build including

smaller wheels to fit within PyPI quotas
unified and simpler build scripts
a new wheel for packagedcode

Tasks

Reviewed contribution guidelines
PR is descriptively titled 📑 and links the original issue above 🔗
Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
Run tests locally to check for errors.
Commits are in uniquely-named feature branch and has no merge conflicts 📁
Updated documentation pages (if applicable)
Updated CHANGELOG.rst (if applicable)

pombredanne added 12 commits

April 26, 2024 17:30


          Switch to pyproject.toml using flot #3459

0a746f3

We are now using only a pyproject.toml

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Introduce new pyproject for packaged code #3459

e45c74b

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Split license data/indexin in new wheels #3459

b13df3d

We now have tow new packages that will be pubslihed as wheels:
* licensedcode-data that contains only the plain text files for
  licenses and license rules
* licensedcode-index that contains only the pre-built index
  with compressed licenses and license rules baked-in

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Add CHANGELOG for new packaging #3459

98cbc93

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Use proper scripts for release #3459

f87d920

Remove old mumbo jumbo renaming manifests

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Add README.rst to license data packaging #3459

00b5f77

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Improve license data packaging #3459

1cd163b

Actually make this works for building with flot.

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Ensure flot is always available #3459

286c272

Since we use pip with --no-build-isolation we need to provide
flot before we install anything.

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          No longer include license data #3459

7fb0a10

These are now provided by the separate licensecode-data and
licensecode-index packages.

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Adjust configure script to use flot #3459

042d2d5

We must install flot first before doing any other installations.
Also cleanup dead code.

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Do not include tests in builds #3459

520685e

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Adjust release scripts for flot #3459

4860e92

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

pombredanne mentioned this pull request

ScanCode contains too many data files #3049

Open

pombredanne added 17 commits

May 6, 2024 12:36


          Use explicit pyproject for ScanCode builds

d5f3569

The plain pyproject.toml is for development only

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Refine packagedcode packaging

5c7bd64

- Remove dependencies that are not direct.

- Start to abstract some of the dependencies on the
  licensedcode module.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Refine packagedcode packaging

2b1d2e7

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Build only modesl in packagedcode

80c72bc

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Move packagedcode.__init__.py code to handlers.py

064c646

Move packagedcode.__init__.py code to handlers.py to allow distributing
multiple smaller packages built with flot.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Split BSD-licensed code in own module

70c5116

This is cleaner and makes tracking licenses much easier.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Improve formattings and docstrings

7e0e5b5

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Guard imports of external packages

451d56a

Guard import with "try/except ImportError" towards packaging models
as a standalone wheel

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Remove unused import

46cb7e8

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Package packagedcode as two wheels

8d780b7

* packagedcode-models only contains the models.py
* packagedcode-pypi only contains the pypi code and depends on models.

This way we have small minimal wheel needed to support FetchCode and
python-inspector.

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Position comment before the commented section

94f32ce

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Build scancode and packagedcode for Pypi

b33a6fc

* Use new pyproject TOML fiels for scancode
* Build packagedcode-models and packagedcode-pypi

Reference: #3459
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Import prospectively license-expression

c64d47d

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Merge latest develop

383318a

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Ensure all licensing usage are guarded

ff47535

Guard the usage also when doing plain expression hanlding that rely on
the SPDX license key data from the LicenseIndex/License cache.

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Update changelog for packagedcode

3a04ec5

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>


          Document vendored flot

67b7ef7

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment