New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch PyProject to Poetry #2111
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just a comment to start a discussion:
I personally use the Poetry package manager instead of directly using pip for projects like this. It has a slightly different format from what setuptools accepts; you can view it here: https://python-poetry.org/docs/pyproject/
I find its dependency resolution mechanism better, with features that prevent the addition of dependencies that are not compatible with others in the project programmatically.
Since in the future we intend to add Sherlock to PyPI, Poetry can also help with that, as it has native build and publish mechanisms, eliminating the need to use other tools like twine.
For the average user, nothing would change; pip would work normally.
For development, well, it has all these features I mentioned above and a few more.
What do you think about this?
Refs:
Personally indifferent. No reservations or preference either way.
|
So, I may be wrong on this, but it seems that poetry does not support dynamic versioning OOTB. The package version will either have to be manually updated each time, or it would require an additional plugin that versions based on git tags (which sherlock doesn't currently use). Thoughts? |
It also doesn't seem that poetry has support for dynamic loading of dependencies -- they would need to be explicitly listed in the pyproj. Thoughts there? Seems finicky on my end. I'm able to install with poetry right now, but unable to uninstall it with poetry (need to use pip). I'm also unable to run sherlock via Latest commit cb22902 is what I have for poetry. |
Rebased onto master to fix conflict (some ~183 behind). Poetry rewritten in a way that actually works. Discovered that the current version of pyproject has some ~undesirable~ install issues, namely it doesn't install the src into a named directory and dumps them straight into site-packages for some unknown reason. Not only is this not pretty, it can cause some conflicts. This version fixes that behavior. Due to how poetry handles packaging, the imports had to be rewritten. Due to how the imports were rewritten, the (already poorly located) unit tests no longer function as desired. Unit tests were relocated to the root where they probably should be anyways. Unit tests work from this location with modified imports. **** PR action currently uses the old unit test location, so failure here is expected. **** GitHub Actions for PRs will be fixed in another PR. I'll probably take advantage of that fixup to switch from the unittest module to tox, which is a bit more portable in my opinion. This also removes the Removed the setup.py and setup.cfg items as they are no longer necessary with the new layout. Removed the rpm spec file. Dynamic versioning still needs to be set up. |
(a couple minor doc updates included, i.e. pyproj readme logo and capitalization) |
Dynamic versioning added but not yet enabled. Relies on named tags and applies throughout Sherlock, rather than pulling from src. |
Fixed an import issue in entry point __main__. Note that $ python -m sherlock --version
Sherlock 0.14.4
Requests 2.28.2
Python 3.12.3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, I may be wrong on this, but it seems that poetry does not support dynamic versioning OOTB.
The package version will either have to be manually updated each time, or it would require an additional plugin that versions based on git tags (which sherlock doesn't currently use). Thoughts?
Yes, poetry doesn't natively support dynamic versioning. We use community plugins for that. We don't need to rely on git tags; we can use the __version__
variable which should be located in sherlock/__init__.py
(this is the location used by the community to store package metadata like version, name, etc.).
The plugin we can use for this is: https://github.com/tiangolo/poetry-version-plugin
It also doesn't seem that poetry has support for dynamic loading of dependencies -- they would need to be explicitly listed in the pyproj. Thoughts there?
This isn't necessary. They should be defined directly in the dependencies group of the project in pyproject.toml
. I see that you've already done that, good job.
Seems finicky on my end. I'm able to install with poetry right now, but unable to uninstall it with poetry (need to use pip). I'm also unable to run sherlock via sherlock due to an odd import error, needing to run it with poetry run sherlock instead. I'm sure both problems can be solved, just haven't seen why yet.
Poetry is a package manager like pip
and also an environment manager like virtualenv
. To use it correctly, you need to activate the virtual environment by using poetry shell
in the project directory.
The command poetry run script.py
does exactly that but does not keep the virtual environment active in the shell session. The virtual environment will only be used to run the script and then it will exit.
Most of my original comments seem to be resolved, which is nice. It was easier to just rewrite it rather than adapt it for some reason.
My original find was Only problem I've noticed is that there doesn't seem to be a way to require your plugin for builds. Since there is no backend, it also doesn't change how things are built sans- Any suggestions to enforce this plugin, or just let people figure it out? |
There is no way to require poetry plugins because they are not part of the project itself but rather part of poetry. This is a community need, and there are discussions about implementing this in the future. Ideally, we should have a workflow for build and publish so we don't handle this manually. For those who want to build locally from the source for any reason, this should be documented in the respective section of the documentation. |
Well in that case, that's all I've got. Probably good to merge... don't think anything else needs to be tweaked (except the github actions, which should probably be addressed separately) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here are a few more suggestions.
Co-authored-by: Matheus Felipe <matheusfelipeog@gmail.com>
Co-authored-by: Matheus Felipe <matheusfelipeog@gmail.com>
Fixes conflict caused by switch to Poetry in sherlock-project/sherlock#2111
This reverts commit 606743b.
Co-authored-by: Matheus Felipe <matheusfelipeog@gmail.com>
# Removing __version__ here will trigger update message for users | ||
# Do not remove until ready to trigger that message | ||
__version__ = "0.14.4" | ||
del __version__ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since this variable is only being used for checking for updates, cant we just remove this and then check the __init__,py
file instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could (and I forgot to make that change, so good call), but we still have to leave that text in place until ready to bump the version
If we remove it from sherlock.py now, before the next version num is ready, it'll trigger unnecessary update available notifications
Will commit a change to pull version num from __init__ in a bit
Continuation of #2036
Package info
Sherlock Project added as Author
Homepage URL set to the github[.]io homepage (PyPi homepage already set, but not pip show)
Dynamic versioning fixed (pulled from sherlock[.]py)
Maintainers added
Classifiers added
Keywords added
Description added
etc
Refocused PR : PrProject being merged elsewhere to expedite package review process and resolve dependency issues (it takes time). This PR is now only to discuss switching to Poetry. (main pr #2116)
::: >>> Jump to start of refocused discussion <<< :::