Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use ruff as linter in pre-commit and GH autotest #6538

Closed
rdimaio opened this issue Mar 7, 2024 · 0 comments
Closed

Use ruff as linter in pre-commit and GH autotest #6538

rdimaio opened this issue Mar 7, 2024 · 0 comments
Assignees
@rdimaio
Labels
enhancement Testing
Milestone
34.1.0

Comments

@rdimaio
Copy link
Contributor

rdimaio commented Mar 7, 2024

ruff is a Python linter that seems to be very widely used, fast and configurable. It replaces flake8, isort, and other tools.

I think it might be worth adding ruff in the pre-commit hook (https://github.com/astral-sh/ruff-pre-commit), instead of isort (#6532), and configuring it to our needs.

Additionally, in the GH autotest workflow (https://github.com/rucio/rucio/blob/master/.github/workflows/autotest.yml), we could replace add_header, python_annotations and python_pyright with either pre-commit/action (https://github.com/pre-commit/action?tab=readme-ov-file) or https://pre-commit.ci/.

  • Reasons for this:
    • Avoid code duplication between checks written in the precommit config and in autotest.yml - we can just point autotest.yml to run the checks in the precommit config, keeping them "in sync"
    • It should be quicker, as it should only run on files changed (and eventual imports followed by the type checker)
@rdimaio rdimaio mentioned this issue Mar 18, 2024
Closed
3 tasks
@rdimaio rdimaio self-assigned this Mar 18, 2024
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Sort imports; rucio#6538
4982042
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Add isort-only Ruff to pre-commit and autotest; rucio#6538
b1debb9 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
@rdimaio rdimaio mentioned this issue Mar 18, 2024
Merged
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove utf-8 declaration from add_header script; rucio#6538
ad19409 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove utf-8 declarations; rucio#6538
4c2bc29 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: pyupgrade, remove unnecessary object inheritance; rucio#6538
36b79df 
Since Python 3, all classes inherit from object by default,
so object can be omitted from the list of base classes.

See also:
- https://docs.astral.sh/ruff/rules/useless-object-inheritance/
- https://peps.python.org/pep-3115/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: pyupgrade, replace exceptions that alias to OSError; rucio#6538
8decbdf 


In Python3.3+, a variety of exceptions (such as IOError) were aliased
to OSError, with the latter being the preferred exception to use.

See also:
- https://docs.python.org/3/library/exceptions.html#OSError
- https://stackoverflow.com/questions/29347790/difference-between-ioerror-and-oserror
- https://docs.astral.sh/ruff/rules/os-error-alias/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove extra parentheses when not needed; rucio#6538
a751689 
See also:
- UP034
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove unnecessary class parentheses; rucio#6538
b9821d2 
See:
- https://docs.astral.sh/ruff/rules/unnecessary-class-parentheses/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove unnecessary str() casting; rucio#6538
a322f62 
See:
- https://docs.astral.sh/ruff/rules/native-literals/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Remove unnecessary encode to utf-8; rucio#6538
1b62daa 
UTF-8 is the default encoding in Python,
so there is no need to call encode when UTF-8 is the desired encoding.
Instead, use a bytes literal.

See also:
- https://docs.astral.sh/ruff/rules/unnecessary-encode-utf8/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: pyupgrade, remove unnecessary import from future; rucio#6538
80696de 
See also:
- https://docs.astral.sh/ruff/rules/unnecessary-future-import/
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Add pyupgrade linting; rucio#6538
9643194 
Pyupgrade is a linter to check for deprecated Python syntax.
This commit configures ruff to perform pyupgrade linting.
In the ignore list, I added some pyupgrade error codes (UP*) for the
moment.
These must either be fixed manually, or might be controversial in some
way, so we can consider them at a later point.
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 18, 2024
@rdimaio
Testing: Extend linter to Python files without extension (e.g. bin/) …
93b341f 
…and lint them; rucio#6538
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Sort imports; rucio#6538
bfc6835
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Add isort-only Ruff to pre-commit and autotest; rucio#6538
0f0aef3 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Sort imports; rucio#6538
6f80d09
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Add isort-only Ruff to pre-commit and autotest; rucio#6538
79bd1fa 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Sort imports; rucio#6538
f4de84e
rdimaio added a commit to rdimaio/rucio that referenced this issue Mar 19, 2024
@rdimaio
Testing: Add isort-only Ruff to pre-commit and autotest; rucio#6538
af3f2a0 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
bari12 pushed a commit that referenced this issue Mar 20, 2024
@rdimaio @bari12
Testing: Sort imports; #6538
03630bc
bari12 pushed a commit that referenced this issue Mar 20, 2024
@rdimaio @bari12
Testing: Add isort-only Ruff to pre-commit and autotest; #6538
1ab4bfc 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
bari12 pushed a commit that referenced this issue Mar 20, 2024
@rdimaio @bari12
Testing: Add isort-only Ruff to pre-commit and autotest; #6538
9ba9770 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
bari12 pushed a commit that referenced this issue Mar 20, 2024
@rdimaio @bari12
Testing: Sort imports; #6538
0288f59
@bari12 bari12 changed the title Consider using ruff as linter in pre-commit and GH autotest USe ruff as linter in pre-commit and GH autotest Mar 20, 2024
@bari12 bari12 changed the title USe ruff as linter in pre-commit and GH autotest Use ruff as linter in pre-commit and GH autotest Mar 20, 2024
@bari12 bari12 added this to the 34.1.0 milestone Mar 20, 2024
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Extend linter to Python files without extension (e.g. bin/) …
bc8fae0 
…and lint them; #6538
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Mark rucio as first-party dependency for isort and re-sort imports; #…
bed3f05 
…6538

rucio is detected as a third-party dependency by default
(likely has to do with the project structure or __init__ files).
This change configures isort to treat rucio as a first-party dependency,
and re-formats the files.
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Remove utf-8 declarations; #6538
6712aed 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: pyupgrade, remove unnecessary object inheritance; #6538
389c996 
Since Python 3, all classes inherit from object by default,
so object can be omitted from the list of base classes.

See also:
- https://docs.astral.sh/ruff/rules/useless-object-inheritance/
- https://peps.python.org/pep-3115/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: pyupgrade, replace exceptions that alias to OSError; #6538
c12826a 
In Python3.3+, a variety of exceptions (such as IOError) were aliased
to OSError, with the latter being the preferred exception to use.

See also:
- https://docs.python.org/3/library/exceptions.html#OSError
- https://stackoverflow.com/questions/29347790/difference-between-ioerror-and-oserror
- https://docs.astral.sh/ruff/rules/os-error-alias/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Remove extra parentheses when not needed; #6538
418783f 
See also:
- UP034
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Remove unnecessary class parentheses; #6538
195edd1 
See:
- https://docs.astral.sh/ruff/rules/unnecessary-class-parentheses/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Remove unnecessary str() casting; #6538
e882313 
See:
- https://docs.astral.sh/ruff/rules/native-literals/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Remove unnecessary encode to utf-8; #6538
758bf1d 
UTF-8 is the default encoding in Python,
so there is no need to call encode when UTF-8 is the desired encoding.
Instead, use a bytes literal.

See also:
- https://docs.astral.sh/ruff/rules/unnecessary-encode-utf8/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: pyupgrade, remove unnecessary import from future; #6538
a075668 
See also:
- https://docs.astral.sh/ruff/rules/unnecessary-future-import/
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Add pyupgrade linting; #6538
75569be 
Pyupgrade is a linter to check for deprecated Python syntax.
This commit configures ruff to perform pyupgrade linting.
In the ignore list, I added some pyupgrade error codes (UP*) for the
moment.
These must either be fixed manually, or might be controversial in some
way, so we can consider them at a later point.
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Testing: Extend linter to Python files without extension (e.g. bin/) …
43ba29d 
…and lint them; #6538
bari12 pushed a commit that referenced this issue Mar 21, 2024
@rdimaio @bari12
Mark rucio as first-party dependency for isort and re-sort imports; #…
405b54f 
…6538

rucio is detected as a third-party dependency by default
(likely has to do with the project structure or __init__ files).
This change configures isort to treat rucio as a first-party dependency,
and re-formats the files.
@bari12 bari12 closed this as completed Apr 2, 2024
rdimaio added a commit to rdimaio/rucio that referenced this issue Apr 2, 2024
@rdimaio
Testing: Add security linting; rucio#6538
eac33ff
bari12 pushed a commit that referenced this issue Apr 2, 2024
@rdimaio @bari12
Testing: Remove utf-8 declaration from add_header script; #6538
093a9be 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Sort imports; rucio#6538
be358b6
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Add isort-only Ruff to pre-commit and autotest; rucio#6538
b085c68 
Adding Ruff to both the pre-commit script
(with --fix passed, thus sorting imports automatically)
and to GH autotest (checking only).
Ruff is currently only configured to check for import sorting.
Other linting functions (flake8, pyupgrade) will be added progressively.
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove utf-8 declaration from add_header script; rucio#6538
ef34ef9 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove utf-8 declarations; rucio#6538
e47b811 
UTF-8 is the default source encoding in Python 3.
Declaring it is unncessary, unless Python 2 is also supported.
See also:
- https://peps.python.org/pep-3120/
- https://docs.python.org/3/howto/unicode.html#python-s-unicode-support
- https://stackoverflow.com/questions/14083111/should-i-use-encoding-declaration-in-python-3
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: pyupgrade, remove unnecessary object inheritance; rucio#6538
0d72db0 
Since Python 3, all classes inherit from object by default,
so object can be omitted from the list of base classes.

See also:
- https://docs.astral.sh/ruff/rules/useless-object-inheritance/
- https://peps.python.org/pep-3115/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: pyupgrade, replace exceptions that alias to OSError; rucio#6538
db75a61 


In Python3.3+, a variety of exceptions (such as IOError) were aliased
to OSError, with the latter being the preferred exception to use.

See also:
- https://docs.python.org/3/library/exceptions.html#OSError
- https://stackoverflow.com/questions/29347790/difference-between-ioerror-and-oserror
- https://docs.astral.sh/ruff/rules/os-error-alias/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove extra parentheses when not needed; rucio#6538
c8a2f5b 
See also:
- UP034
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove unnecessary class parentheses; rucio#6538
ced23e5 
See:
- https://docs.astral.sh/ruff/rules/unnecessary-class-parentheses/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove unnecessary str() casting; rucio#6538
1b58ad0 
See:
- https://docs.astral.sh/ruff/rules/native-literals/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Remove unnecessary encode to utf-8; rucio#6538
ea8924f 
UTF-8 is the default encoding in Python,
so there is no need to call encode when UTF-8 is the desired encoding.
Instead, use a bytes literal.

See also:
- https://docs.astral.sh/ruff/rules/unnecessary-encode-utf8/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: pyupgrade, remove unnecessary import from future; rucio#6538
c373289 
See also:
- https://docs.astral.sh/ruff/rules/unnecessary-future-import/
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Add pyupgrade linting; rucio#6538
698e941 
Pyupgrade is a linter to check for deprecated Python syntax.
This commit configures ruff to perform pyupgrade linting.
In the ignore list, I added some pyupgrade error codes (UP*) for the
moment.
These must either be fixed manually, or might be controversial in some
way, so we can consider them at a later point.
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Testing: Extend linter to Python files without extension (e.g. bin/) …
94e8889 
…and lint them; rucio#6538
voetberg pushed a commit to voetberg/rucio that referenced this issue Apr 15, 2024
@rdimaio @voetberg
Mark rucio as first-party dependency for isort and re-sort imports; r…
146c50b 
…ucio#6538

rucio is detected as a third-party dependency by default
(likely has to do with the project structure or __init__ files).
This change configures isort to treat rucio as a first-party dependency,
and re-formats the files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rdimaio rdimaio

Labels
enhancement Testing
Projects
None yet
Milestone
34.1.0
Development

No branches or pull requests
2 participants
@bari12 @rdimaio