Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use setuptools to install python library on Mac #219

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

alice
Copy link
Contributor

@alice alice commented May 3, 2024

Adds a custom install_python target that builds and installs a python library named acacia_axapi.

@spectranaut
Copy link
Contributor

Hmmm I ran into some problems, I rebuilt main to main (successfully) to make sure it wasn't on my end.

I ran pip install —upgrade build like you mentioned in the chat, but did you forget some other step you had to do along the way...? Here are the errors I get:

When I try to build python:

~/repos/acacia/build  9:31AM> cmake -DACACIA_PYTHON=ON .. -G 'Ninja' --fresh

....

~/repos/acacia/build  9:31AM> ninja                                         
[9/11] cd /Users/valerieyoung/repos/acacia/build/lib/mac && /opt/homebrew/Cellar/cmake/3.27.8/bin/cmake -E copy /Users/valerieyoung/repos...ia/lib/mac/acacia_axapi.i /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi.i && MACOSX_DEPLOYMENT_TARGET=13.0 python3 -m buil
FAILED: lib/mac/CMakeFiles/python_acacia_axapi /Users/valerieyoung/repos/acacia/build/lib/mac/CMakeFiles/python_acacia_axapi 
cd /Users/valerieyoung/repos/acacia/build/lib/mac && /opt/homebrew/Cellar/cmake/3.27.8/bin/cmake -E copy /Users/valerieyoung/repos/acacia/lib/mac/acacia_axapi.i /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi.i && MACOSX_DEPLOYMENT_TARGET=13.0 python3 -m build
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools >= 40.8.0
* Getting build dependencies for sdist...
running egg_info
creating acacia_axapi.egg-info
writing acacia_axapi.egg-info/PKG-INFO
writing dependency_links to acacia_axapi.egg-info/dependency_links.txt
writing top-level names to acacia_axapi.egg-info/top_level.txt
writing manifest file 'acacia_axapi.egg-info/SOURCES.txt'
reading manifest file 'acacia_axapi.egg-info/SOURCES.txt'
writing manifest file 'acacia_axapi.egg-info/SOURCES.txt'
* Building sdist...
running sdist
running egg_info
writing acacia_axapi.egg-info/PKG-INFO
writing dependency_links to acacia_axapi.egg-info/dependency_links.txt
writing top-level names to acacia_axapi.egg-info/top_level.txt
reading manifest file 'acacia_axapi.egg-info/SOURCES.txt'
writing manifest file 'acacia_axapi.egg-info/SOURCES.txt'
warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md

running check
creati`ng acacia_axapi-0.0.0
creating acacia_axapi-0.0.0/acacia_axapi.egg-info
copying files to acacia_axapi-0.0.0...
copying setup.py -> acacia_axapi-0.0.0
copying acacia_axapi.egg-info/PKG-INFO -> acacia_axapi-0.0.0/acacia_axapi.egg-info
copying acacia_axapi.egg-info/SOURCES.txt -> acacia_axapi-0.0.0/acacia_axapi.egg-info
copying acacia_axapi.egg-info/dependency_links.txt -> acacia_axapi-0.0.0/acacia_axapi.egg-info
copying acacia_axapi.egg-info/top_level.txt -> acacia_axapi-0.0.0/acacia_axapi.egg-info
copying acacia_axapi.egg-info/SOURCES.txt -> acacia_axapi-0.0.0/acacia_axapi.egg-info
Writing acacia_axapi-0.0.0/setup.cfg
Creating tar archive
removing 'acacia_axapi-0.0.0' (and everything under it)
* Building wheel from sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools >= 40.8.0
* Getting build dependencies for wheel...
running egg_info
writing acacia_axapi.egg-info/PKG-INFO
writing dependency_links to acacia_axapi.egg-info/dependency_links.txt
writing top-level names to acacia_axapi.egg-info/top_level.txt
reading manifest file 'acacia_axapi.egg-info/SOURCES.txt'
writing manifest file 'acacia_axapi.egg-info/SOURCES.txt'
* Installing packages in isolated environment:
  - wheel
* Building wheel...
running bdist_wheel
running build
running build_ext
building '_acacia_axapi' extension
swigging /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi.i to /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi_wrap.cpp
swig -python -c++ -I/Users/valerieyoung/repos/acacia/include -o /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi_wrap.cpp /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi.i
creating build
creating build/temp.macosx-14-arm64-cpython-311
creating build/temp.macosx-14-arm64-cpython-311/Users
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos/acacia
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos/acacia/build
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos/acacia/build/lib
creating build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos/acacia/build/lib/mac
clang -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/Users/valerieyoung/repos/acacia/include -I/private/var/folders/vf/8syk3pc15m5_3tp47tfb0x_m0000gn/T/build-env-k7ofr9rq/include -I/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11 -c /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi_wrap.cpp -o build/temp.macosx-14-arm64-cpython-311/Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi_wrap.o -std=c++11
In file included from /Users/valerieyoung/repos/acacia/build/lib/mac/acacia_axapi_wrap.cpp:168:
/opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/include/python3.11/Python.h:23:12: fatal error: 'stdlib.h' file not found
   23 | #  include <stdlib.h>
      |            ^~~~~~~~~~
1 error generated.
error: command '/Users/valerieyoung/repos/chromium/src/third_party/llvm-build/Release+Asserts/bin/clang' failed with exit code 1

ERROR Backend subprocess exited when trying to invoke build_wheel
ninja: build stopped: subcommand failed.

Also btw when I try to build the nodejs, because of your updates to the swig interface file:

~/repos/acacia/build  9:28AM> cmake -DACACIA_NODEJS=ON .. -G 'Ninja' --fresh 

.....

~/repos/acacia/build  9:28AM> ninja
[5/11] Generating nodejs_acacia_axapi_wrap.cxx
FAILED: lib/mac/nodejs_acacia_axapi_wrap.cxx /Users/valerieyoung/repos/acacia/build/lib/mac/nodejs_acacia_axapi_wrap.cxx 
cd /Users/valerieyoung/repos/acacia/build/lib/mac && swig -c++ -javascript -node -o nodejs_acacia_axapi_wrap.cxx -I/Users/valerieyoung/repos/acacia /Users/valerieyoung/repos/acacia/lib/mac/acacia_axapi.i
/Users/valerieyoung/repos/acacia/lib/mac/acacia_axapi.i:53: Error: Unable to find 'acacia/mac/mac_data_types.h'
/Users/valerieyoung/repos/acacia/lib/mac/acacia_axapi.i:54: Error: Unable to find 'acacia/mac/axapi_node.h'
[7/11] Building CXX object examples/mac/CMakeFiles/attribute_types.dir/attribute_types.cc.o
ninja: build stopped: subcommand failed

@alice
Copy link
Contributor Author

alice commented May 6, 2024

Yeah, I was expecting the nodejs build to break - I'll look into that before marking this for review.

I'm not able to reproduce the issue you're having, though, even after removing and reinstalling (as best I can) the Python framework.

I tried using Homebrew python to match your setup (I'm using the official install from https://www.python.org/downloads/), but then straightforwardly trying to run pip3 install --upgrade build didn't work and wanted me to set up a virtual environment:

% pip3 install build
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try brew install
    xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a Python library that isn't in Homebrew,
    use a virtual environment:
    
    python3 -m venv path/to/venv
    source path/to/venv/bin/activate
    python3 -m pip install xyz
    
    If you wish to install a Python application that isn't in Homebrew,
    it may be easiest to use 'pipx install xyz', which will manage a
    virtual environment for you. You can install pipx with
    
    brew install pipx
    
    You may restore the old behavior of pip by passing
    the '--break-system-packages' flag to pip, or by adding
    'break-system-packages = true' to your pip.conf file. The latter
    will permanently disable this error.
    
    If you disable this error, we STRONGLY recommend that you additionally
    pass the '--user' flag to pip, or set 'user = true' in your pip.conf
    file. Failure to do this can result in a broken Homebrew installation.
    
    Read more about this behavior here: <https://peps.python.org/pep-0668/>

I tried brew install python-build, but that didn't seem to work when I tried to build (apologies, I didn't save the error message from that).

So I'm not sure how to faithfully reproduce your build environment, sorry :(

@alice
Copy link
Contributor Author

alice commented May 8, 2024

@spectranaut With any luck, this is now actually working. Would you be able to try running it on your Mac when you have a chance?

To install, one you have build installed, you'll "just" need to run

ninja install_python

This is a separate target from the rest of the install targets because you don't necessarily want to run pip install as sudo.

If it's working for you, then I think it's ready for a review!

@alice alice requested a review from spectranaut May 8, 2024 07:56
@alice alice marked this pull request as ready for review May 8, 2024 07:57
@@ -45,21 +45,3 @@ if (ACACIA_NODEJS)
VERBATIM
)
endif (ACACIA_NODEJS)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the library is now installed globally, we no longer need to make sure we copy the library next to the script.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about someone who wants to develop and test but not install the library -- it seems to me that the install (ninja install) and build (ninja) should be different commands? In the build case, we should still do this step, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm that's a good point :) I'll revert this change, but later on perhaps as part of investigating virtual environments we could see whether there's a way to set up a venv for the example script.

examples/mac/dump_tree_axapi.py Show resolved Hide resolved
lib/mac/CMakeLists.txt Show resolved Hide resolved

[project]
name = 'acacia_axapi'
version = '0.0.1'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should eventually add a top-level metadata file with version, authors, license etc. but this will do for now.

lib/mac/CMakeLists.txt Outdated Show resolved Hide resolved
@alice alice changed the title First pass getting python library installed on Mac using setuptools Use setuptools to install python library on Mac May 8, 2024
lib/mac/CMakeLists.txt Outdated Show resolved Hide resolved
lib/mac/CMakeLists.txt Outdated Show resolved Hide resolved
Copy link
Contributor

@spectranaut spectranaut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yay, success! It works for me! Curious about a few things.


```
brew install cmake
brew install ninja
brew install swig
```

If you encounter a python3_LIBRARIES not found, try updating python with:
Ensure you have Python frameworks installed. You can install Python using `brew`:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is just for the python bindings, we probably should say.

Maybe we should have a ##### Python bindings sub header here, and nodejs below?

README.md Show resolved Hide resolved
@@ -45,21 +45,3 @@ if (ACACIA_NODEJS)
VERBATIM
)
endif (ACACIA_NODEJS)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about someone who wants to develop and test but not install the library -- it seems to me that the install (ninja install) and build (ninja) should be different commands? In the build case, we should still do this step, right?

examples/mac/dump_tree_axapi.py Show resolved Hide resolved
lib/mac/CMakeLists.txt Show resolved Hide resolved
lib/mac/CMakeLists.txt Show resolved Hide resolved
@spectranaut
Copy link
Contributor

We should make a follow up issue to install the node module, right?

Copy link
Contributor

@spectranaut spectranaut left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so I ran this with the hopes we could finally land it, but even despite your revert, when I try to run build via "ninja" then run the example, it doesn't work:

~/repos/acacia/build  2:02PM> python3 ./examples/mac/dump_tree_axapi.py
Traceback (most recent call last):
  File "/Users/valerieyoung/repos/acacia/build/./examples/mac/dump_tree_axapi.py", line 5, in <module>
    import acacia_axapi
ModuleNotFoundError: No module named 'acacia_axapi'

But it turns out this is failing on main to! so we need to investigate. I'll make an issue. Otherwise this is approved.

lib/mac/CMakeLists.txt Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants