Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python 3.11.5 ERROR: Could not build wheels for pypika, #761

Open
pauldintel opened this issue Oct 10, 2023 · 12 comments
Open

Python 3.11.5 ERROR: Could not build wheels for pypika, #761

pauldintel opened this issue Oct 10, 2023 · 12 comments

Comments

@pauldintel
Copy link

Any solution to this ?

sudo pip3 install pypika
sudo pip install pypika

both throws below error :

Building wheels for collected packages: pypika
Building wheel for pypika (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for pypika (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [86 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/pypika
copying pypika/enums.py -> build/lib/pypika
copying pypika/queries.py -> build/lib/pypika
copying pypika/utils.py -> build/lib/pypika
copying pypika/functions.py -> build/lib/pypika
copying pypika/pseudocolumns.py -> build/lib/pypika
copying pypika/dialects.py -> build/lib/pypika
copying pypika/analytics.py -> build/lib/pypika
copying pypika/init.py -> build/lib/pypika
copying pypika/terms.py -> build/lib/pypika
creating build/lib/pypika/clickhouse
copying pypika/clickhouse/nullable_arg.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/type_conversion.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/search_string.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/init.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/array.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/dates_and_times.py -> build/lib/pypika/clickhouse
copying pypika/clickhouse/condition.py -> build/lib/pypika/clickhouse
running egg_info
warning: no files found matching 'requirements.txt'
writing manifest file 'PyPika.egg-info/SOURCES.txt'
running install
running install_lib
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in
main()
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 268, in run_setup
self).run_setup(setup_script=setup_script)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 33, in
setup(
File "/usr/lib/python3/dist-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 335, in run
self.run_command('install')
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/setuptools/command/install.py", line 68, in run
return orig.install.run(self)
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/install.py", line 622, in run
self.run_command(cmd_name)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 985, in run_command
cmd_obj.ensure_finalized()
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/usr/lib/python3/dist-packages/setuptools/command/install_lib.py", line 17, in finalize_options
self.set_undefined_options('install',('install_layout','install_layout'))
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 290, in set_undefined_options
setattr(self, dst_option, getattr(src_cmd_obj, src_option))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in getattr
raise AttributeError(attr)
AttributeError: install_layout. Did you mean: 'install_platlib'?
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pypika
Failed to build pypika
ERROR: Could not build wheels for pypika, which is required to install pyproject.toml-based projects

@wd60622
Copy link
Contributor

wd60622 commented Oct 12, 2023

I think that the current pyproject.toml needs to adopt the new standard. It may look like this: https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
How have you installed python?

Does it work with any other python 3.11 versions?
update: I was able to install on python 3.11.5

FROM python:3.11.5

RUN pip install pypika
docker build -t pypika-3.11.5 .

@pauldintel
Copy link
Author

hi @wd60622 -

Also it seem with python 3.10.12 no isssue with pypika however 3.11 has issue and until I have 3.11 I can't proceed with pip install vectordb-bench . so its a catch 22 situations ..

$ pip install vectordb-bench
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement vectordb-bench (from versions: none)
ERROR: No matching distribution found for vectordb-bench

$ python3 -V
Python 3.10.12
$ pip install PyPika
Defaulting to user installation because normal site-packages is not writeable
Collecting PyPika
Downloading PyPika-0.48.9.tar.gz (67 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 67.3/67.3 KB 2.8 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: PyPika
Building wheel for PyPika (pyproject.toml) ... done
Created wheel for PyPika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53738 sha256=ff1cb26b6fb29882d18bf1894abdb2b26c3e21d033d4592cf42901488eeb4f41
Stored in directory: /home/ubuntu/.cache/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4
Successfully built PyPika
Installing collected packages: PyPika
Successfully installed PyPika-0.48.9
:~/ann-benchmarks$ cd ..

@pauldintel
Copy link
Author

this is problem with pypika being on vectordb-bench and I raised the issue here as well : zilliztech/VectorDBBench#224

But at this moment I need solution how can I move from this deadlock situation please

@wd60622
Copy link
Contributor

wd60622 commented Oct 12, 2023

Interesting

I am able to create an environment with those two packages

FROM python:3.11.5

RUN pip install pypika vectordb-bench

How have you installed python and what type of machine are you using? I think this is an indication of system dependencies. Similar error message: https://stackoverflow.com/questions/70508775/error-could-not-build-wheels-for-pycairo-which-is-required-to-install-pyprojec

I think a way to work around this is to run your application in a docker container

@pauldintel
Copy link
Author

I have Ubuntu 22.04 shipped with Python 3.10.12 and updated this to 3.11.5 using below instructions in AWS m7i/m6i Intel x86/64 instances :
https://www.itsupportwale.com/blog/how-to-upgrade-to-python-3-11-on-ubuntu-20-04-and-22-04-lts/

And then did : pip install pypika vectordb-bench

ending with error :

AttributeError: install_layout. Did you mean: 'install_platlib'?
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pypika
Failed to build pypika
ERROR: Could not build wheels for pypika, which is required to install pyproject.toml-based projects

I saw that stackworkflow thread but can't get any pointer to solve this .
per vectordb advice I am running this application and not within container .

@pauldintel
Copy link
Author

pauldintel commented Oct 12, 2023

I ran a sample Dockerfile and concur with you that yes it strangely works inside the container .. question is why

$pip3 list | grep vector
pgvector 0.2.3
vectordb-bench 0.0.5

$python3 -V
Python 3.11.5

$pip3 list | grep PyPika
PyPika 0.48.9

@AzisK
Copy link

AzisK commented Oct 13, 2023

I tried to use python:3.11.5 Docker image and it failed installing the libraries from the first time. The error was different but a little similar (I don't have it by now). I was not able to reproduce it. It is surprising to me to not be able to reproduce the same behavior using a Docker container. Could it be because of the caching of pip libraries that I used?

Then I tried different images slim, bullseye, slim-bullseye, alpine and all of them worked. Then I used specifically Ubuntu v22.04 image and installed Python there. The image built successfully again. I am sharing my Dockerfile with the hope that it helps to find the issue.

# syntax=docker/dockerfile:1

# Comments are provided throughout this file to help you get started.
# If you need more help, visit the Dockerfile reference guide at
# https://docs.docker.com/engine/reference/builder/

FROM ubuntu:22.04 as base

ARG PYTHON_VERSION=3.11.5
# Install build dependencies for Python ${PYTHON_VERSION}
RUN apt-get update && apt-get install -y --no-install-recommends \
    build-essential \
    wget \
    libssl-dev \
    zlib1g-dev \
    libncurses5-dev \
    libgdbm-dev \
    libnss3-dev \
    libreadline-dev \
    libffi-dev \
    libsqlite3-dev \
    libbz2-dev

# Download Python ${PYTHON_VERSION} source code
WORKDIR /tmp
RUN wget --no-check-certificate https://www.python.org/ftp/python/${PYTHON_VERSION}/Python-${PYTHON_VERSION}.tgz

# Extract the source code
RUN tar -xf Python-${PYTHON_VERSION}.tgz

# Build and install Python ${PYTHON_VERSION}
WORKDIR /tmp/Python-${PYTHON_VERSION}
RUN ./configure --enable-optimizations
RUN make -j$(nproc)
RUN make install

# Cleanup
WORKDIR /
RUN rm -rf /tmp/Python-${PYTHON_VERSION}

# Prevents Python from writing pyc files.
ENV PYTHONDONTWRITEBYTECODE=1

# Keeps Python from buffering stdout and stderr to avoid situations where
# the application crashes without emitting any logs due to buffering.
ENV PYTHONUNBUFFERED=1

WORKDIR /app

# Download dependencies as a separate step to take advantage of Docker's caching.
# Leverage a cache mount to /root/.cache/pip to speed up subsequent builds.
# Leverage a bind mount to requirements.txt to avoid having to copy them into
# into this layer.
RUN --mount=type=cache,target=/root/.cache/pip \
    --mount=type=bind,source=requirements.txt,target=requirements.txt \
    python3 -m pip install -r requirements.txt

# Copy the source code into the container.
COPY . .

# Run the application.
CMD python3 app.py
# requirements.txt file
pypika
vectordb-bench

P.S. I think the error is related to setuptools. This docker container has this version 65.5.0 of setuptools

@wd60622
Copy link
Contributor

wd60622 commented Oct 15, 2023

@pauldintel Did the setuptools version impact the install result?
@AzisK Do you think there can be anything done on PyPika side to make this easier for user?

@AzisK
Copy link

AzisK commented Oct 16, 2023

I guess we could test against different versions of setuptools and having found the ones that do not work we could add a check of setuptools during the installation and then inform the user about the supported setuptools versions. I don't know if this is worth the effort

To my knowledge, I don't know what else can we do

@pauldintel
Copy link
Author

hi @wd60622 I have tried running this inside Python virtualenv and it works perfectly fine . I know as a dockerfile it might work inside the container too . but I am not stuck with virtualenv at this moment . since vectordb (like chormadb) referring to your packages I was stuck with that but I am fine now. But something is wrong w.o venv not sure what is that !

@wd60622
Copy link
Contributor

wd60622 commented Oct 16, 2023

hi @wd60622 I have tried running this inside Python virtualenv and it works perfectly fine . I know as a dockerfile it might work inside the container too . but I am not stuck with virtualenv at this moment . since vectordb (like chormadb) referring to your packages I was stuck with that but I am fine now. But something is wrong w.o venv not sure what is that !

Glad you have it working. I am guessing @AzisK was write with the setuptools version

Can this issue be closed then?

@redfalcoon
Copy link

Also in my environmet with python 3.11.9 (not in a docker environment) I wasn't able to install pypika 0.48.9 but, I've solved the issue with this workaround:

first: reinstall setuptools

pip install --upgrade --force-reinstall setuptools

then reinstall pypika

export SETUPTOOLS_USE_DISTUTILS=stdlib
pip install pypika

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants