use pytoolconfig for configuration

[bagel897]

Description

Fixes #401
Use pyproject.toml based configuration via pytoolconfig.

Behavior is now

1. Check if [tool.rope] is in pyproject.toml. If so, use that configuration
2. Check if config.py exists. If so, use that configuration. (Supports project_opened and callable functions)
3. Use default config specified in pydantic model

Rope will no longer create a config.py, but instead document each configuration key.
All the keys have been placed in rope/base/prefs.py. Future keys must be placed and documented here.

Documentation

The project now uses sphinx to generate documentation (includeing configuration keys). It should work with read the docs out of the box.

Pyproject.toml

I moved setuptools metadata to pyproject.toml
https://setuptools.pypa.io/en/latest/userguide/quickstart.html - use config.py stub

Python 3.6

Python 3.6 is unsupported

• setuptools doesn't work properly with pyproject.toml on python 3.6. (It may have been my fault, I can check)
• tomli doesn't support python3.6 with their latest versions
• python doesn't support python 3.6
• my library, pytoolconfig doesn't support python 3.6 (but this can be fixed if needed)

I will implement the ability for rope to target old python versions automatically in a future PR, but this PR drops 3.6 support.

Pathlib

I added a pathlib method to the resource object to ease use of pathlib. I'm not sure if I used the correct name for it or should remove it.

Checklist (delete if not relevant):

• I have added tests that prove my fix is effective or that my feature works (One test)
• I will resolve the config.py generation issue - doesn't generate it.
• I will add minimum_py_version. This will allow rope to detect the lowest python version used by the project, then target it. This way, rope can be run in up-to-date versions of python (3.7+) and be used on (3.4) - preference added but not used.
• I will add a list of dependencies. This will allow autoimport to only cache modules that are used by the project. Will do after Use multithread executor for local imports, process imports #469
• I have updated CHANGELOG.md
• I have made corresponding changes to user documentation for new features
• I have made corresponding changes to library documentation for API changes - should be api compatible

[lieryan]

Having minimal dependency outside of the standard library is one of the important goal of rope for me. I'd prefer to not have runtime dependencies on pydantic. It's a fairly popular an up-and-coming library and likely going to get more steam in the future, and the nature of this library is that it's likely to be installed as part of user projects. So, if we require pydantic at runtime, there's a pretty good chance that rope's pydantic version requirement conflicting with the user project's pydantic version requirement.

In my ideal world, pytoolconfig would generate the documentation and project metadata from a pydantic model, and the output files and any metadata needed for the config parser generated from the model can be committed into the rope repository. It's ok if rope developers have to install pydantic when developing rope itself, but rope users should not need to install pydantic itself just for using rope.

The other dependencies that we pulled tomli is ok as an exception, because installing tomli is basically just backporting a future standard library module.

Also, if rope depend on pytoolconfig at runtime for config parsing, I'd prefer to avoid that, but it might be ok if pytoolconfig itself stays conservative with its own runtime dependency (i.e. don't require pydantic to be installed at runtime). This would mean pytoolconfig might have to generate boilerplate class so that the config parser code would not directly depend on pydantic. That sounds complicated though, so we might as well just hand-write the toml/cfg config parser.

[bagel897]

pydantic offers a couple of nice features - namely quick configuration validation. An alternative arrangement would be to use python dataclasses, and make pydantic a fully optional runtime dependency for validation. I will look into the viability of this option, since it reduces pytoolconfig's non-standard dependencies to 0 (packaging and tomli are both basically guaranteed to be installed).

[bagel897]

I've successfully removed the pydantic dependency by making pytoolconfig use dataclasses. I've also dropped python 3.6 (and some old python 2 code and <3.6 code) from this branch and gave rope the ability to detect the python version a project is using (and dependencies for autoimport).
Lets say a project is uses python 3.5 to 3.8. Previously, rope would be installed in the same virtual environment. But now we know what version of python that project is using, so we can change the language level AST parses. The next step would be to make that happen, but I'm not familiar enough with ropes internals to know how viable that would be. This way, rope can drop unsupported python versions while being usable on older projects.

Documentation

I made a sphinx generator for configuration documentation and configured sphinx . I didn't add the RTD theme, but otherwise it's possible to use it on readthedocs.

[lieryan]

Hi Bagel, thanks for this, I've cherry-picked some of the commits out of this PR, since they're unrelated to the pytoolconfig/toml changes.

In the future, I'd recommend creating individual PRs for each separate sets of changes, as it will make it much easier to review.

[lieryan]

typo: this should have been perform_doa

[bagel897]

fixed

[bagel897]

Why did you restore the default_config.py?

1. It would eventually go out of date, or we'd have two sets of configuration to maintain
2. Documentation for config.py would be in rst/rtd, which is more user-friendly
3. It isn't written to the ropefolder anymore.

[lieryan]

Why did you restore the default_config.py?

I restored it because we still need to provide an example configuration for people that still want to use config.py.

Eventually I'd like this example file to be automatically generated from the Prefs dataclass so that it doesn't get out of sync with Prefs. I'll need to figure out how to do this in a way that fits within the current setup, but maybe some make html kind of command like Sphinx. The file should probably be moved into docs folder as well as it's no longer intended to be executed at runtime.

[bagel897]

The documentation has documentation on config.py and all the subkeys. I may look into generating a config.py in the future, perhaps standardizing into pytoolconfig somehow. Maybe we can move it into docs and link it from the configuration.rst?

[lieryan]

Maybe we can move it into docs and link it from the configuration.rst?

Yes, that's the plan; it's now no longer the primary form of documentation, but just providing examples that users can copy and adapt and should be linked from configuration.rst. It would be useful to have examples for pyproject.toml as well.

Not quite sure what the API can look like to be standardized in pytoolconfig, but it's basically just going to be "rendering a template based on Prefs dataclass". So either pytoolconfig need to gain a templating engine for this purpose or just keep it simple with a generic callable mechanism that, given the Prefs class, the callback can do anything the application needs to do, like generating files.

[lieryan]

For the scope of this PR though, I think we should just link this file from configuration.rst and move the default_config.py to docs. Auto-generating the example files should be looked into in a separate PR.

[bagel897]

Not quite sure what the API can look like to be standardized in pytoolconfig, but it's basically just going to be "rendering a template based on Prefs dataclass". So either pytoolconfig need to gain a templating engine for this purpose or just keep it simple with a generic callable mechanism that, given the Prefs class, the callback can do anything the application needs to do, like generating files.

Well the original idea was writing a whole markdown file from a PyToolConfig object, which contained all the requisite information for generating an entire configuration file But I decided that a custom markdown build system is a bad idea, so switched to sphinx but don't like their extension system as much. I'll have to look into better ways to do this (and PR changes back here), I think the way I'll go is generate markdown and use it in sphinx via myst.

[lieryan]

Thank you for the really great work in improving and modernizing rope's tooling situation, @bageljrkhanofemus.