Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pylint] - add fix for unary expressions in PLC2801 #9587

Merged
merged 7 commits into from
Mar 4, 2024

Conversation

diceroll123
Copy link
Contributor

Summary

Closes #9572

Don't go easy on this review!

Test Plan

cargo test

Copy link
Contributor

github-actions bot commented Jan 20, 2024

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

ℹ️ ecosystem check detected linter changes. (+48 -0 violations, +0 -48 fixes in 5 projects; 38 projects unchanged)

apache/airflow (+0 -0 violations, +0 -6 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview --select ALL

+ tests/providers/amazon/aws/hooks/test_base_aws.py:556:17: PLC2801 Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
- tests/providers/amazon/aws/hooks/test_base_aws.py:556:17: PLC2801 [*] Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
+ tests/providers/google/common/hooks/test_discovery_api.py:134:17: PLC2801 Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
- tests/providers/google/common/hooks/test_discovery_api.py:134:17: PLC2801 [*] Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
+ tests/providers/google/common/hooks/test_discovery_api.py:140:17: PLC2801 Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
- tests/providers/google/common/hooks/test_discovery_api.py:140:17: PLC2801 [*] Unnecessary dunder call to `__bool__`. Use `bool()` builtin.

aws/aws-sam-cli (+0 -0 violations, +0 -24 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ tests/unit/commands/local/lib/test_debug_context.py:32:25: PLC2801 Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
- tests/unit/commands/local/lib/test_debug_context.py:32:25: PLC2801 [*] Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
+ tests/unit/commands/local/lib/test_debug_context.py:45:26: PLC2801 Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
- tests/unit/commands/local/lib/test_debug_context.py:45:26: PLC2801 [*] Unnecessary dunder call to `__bool__`. Use `bool()` builtin.
+ tests/unit/local/apigw/test_local_apigw_service.py:1995:29: PLC2801 Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
- tests/unit/local/apigw/test_local_apigw_service.py:1995:29: PLC2801 [*] Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
+ tests/unit/local/apigw/test_local_apigw_service.py:1995:48: PLC2801 Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
- tests/unit/local/apigw/test_local_apigw_service.py:1995:48: PLC2801 [*] Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
+ tests/unit/local/apigw/test_local_apigw_service.py:2000:29: PLC2801 Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
- tests/unit/local/apigw/test_local_apigw_service.py:2000:29: PLC2801 [*] Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
+ tests/unit/local/apigw/test_local_apigw_service.py:2000:48: PLC2801 Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
- tests/unit/local/apigw/test_local_apigw_service.py:2000:48: PLC2801 [*] Unnecessary dunder call to `__hash__`. Use `hash()` builtin.
... 12 additional changes omitted for project

freedomofpress/securedrop (+0 -0 violations, +0 -14 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ securedrop/pretty_bad_protocol/_meta.py:677:54: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
- securedrop/pretty_bad_protocol/_meta.py:677:54: PLC2801 [*] Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ securedrop/pretty_bad_protocol/_meta.py:688:59: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
- securedrop/pretty_bad_protocol/_meta.py:688:59: PLC2801 [*] Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ securedrop/pretty_bad_protocol/_util.py:364:11: PLC2801 Unnecessary dunder call to `__str__`. Use `str()` builtin.
- securedrop/pretty_bad_protocol/_util.py:364:11: PLC2801 [*] Unnecessary dunder call to `__str__`. Use `str()` builtin.
+ securedrop/tests/test_db.py:76:9: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
- securedrop/tests/test_db.py:76:9: PLC2801 [*] Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ securedrop/tests/test_db.py:83:9: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
- securedrop/tests/test_db.py:83:9: PLC2801 [*] Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
... 4 additional changes omitted for project

milvus-io/pymilvus (+0 -0 violations, +0 -4 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pymilvus/client/prepare.py:1196:24: PLC2801 Unnecessary dunder call to `__str__`. Use `str()` builtin.
- pymilvus/client/prepare.py:1196:24: PLC2801 [*] Unnecessary dunder call to `__str__`. Use `str()` builtin.
+ pymilvus/orm/iterator.py:247:19: PLC2801 Unnecessary dunder call to `__len__`. Use `len()` builtin.
- pymilvus/orm/iterator.py:247:19: PLC2801 [*] Unnecessary dunder call to `__len__`. Use `len()` builtin.

pandas-dev/pandas (+48 -0 violations, +0 -0 fixes)

ruff check --no-cache --exit-zero --ignore RUF9 --output-format concise --preview

+ pandas/tests/arrays/sparse/test_arithmetics.py:407:14: PLC2801 Unnecessary dunder call to `__add__`. Use `+` operator.
+ pandas/tests/arrays/string_/test_string.py:195:12: PLC2801 Unnecessary dunder call to `__add__`. Use `+` operator.
+ pandas/tests/arrays/string_/test_string.py:211:12: PLC2801 Unnecessary dunder call to `__add__`. Use `+` operator.
+ pandas/tests/base/test_conversion.py:135:28: PLC2801 Unnecessary dunder call to `__iter__`. Use `iter()` builtin.
+ pandas/tests/base/test_conversion.py:53:28: PLC2801 Unnecessary dunder call to `__iter__`. Use `iter()` builtin.
+ pandas/tests/base/test_conversion.py:85:28: PLC2801 Unnecessary dunder call to `__iter__`. Use `iter()` builtin.
+ pandas/tests/dtypes/test_dtypes.py:230:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/groupby/test_groupby.py:2156:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/groupby/test_groupby.py:2159:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/groupby/test_grouping.py:1116:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:105:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:111:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:120:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:153:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:158:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:173:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:51:22: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:63:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:69:18: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:81:22: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/multi/test_formats.py:93:22: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/test_base.py:778:33: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/indexes/test_common.py:202:29: PLC2801 Unnecessary dunder call to `__repr__`. Use `repr()` builtin.
+ pandas/tests/scalar/timedelta/test_arithmetic.py:1156:12: PLC2801 Unnecessary dunder call to `__add__`. Use `+` operator.
+ pandas/tests/scalar/timedelta/test_arithmetic.py:1157:12: PLC2801 Unnecessary dunder call to `__sub__`. Use `-` operator.
... 23 additional changes omitted for project

Changes by rule (1 rules affected)

code total + violation - violation + fix - fix
PLC2801 96 48 0 0 48

@diceroll123
Copy link
Contributor Author

I see now that the set_fix in both sides of the unary if statement can be moved out, but I won't touch it until after you've reviewed @charliermarsh

@charliermarsh
Copy link
Member

@diceroll123 - Feel free to change, I won't get to this until tomorrow.

Copy link

codspeed-hq bot commented Jan 26, 2024

CodSpeed Performance Report

Merging #9587 will not alter performance

Comparing diceroll123:tweak-PLC2801 (1a14221) with main (c007b17)

Summary

✅ 30 untouched benchmarks

@MichaReiser MichaReiser added the fixes Related to suggested fixes for violations label Jan 26, 2024
Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to generalize the fix to correctly handle operator precedence in general. There are some more cases mentioned on the linked Issue that this PR doesn't address.

if let Some(fixed) = fixed {
diagnostic.set_fix(Fix::safe_edit(Edit::range_replacement(fixed, call.range())));
if let Some(mut fixed) = fixed {
let is_in_unary = checker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section here could use a comment explaining on why we're doing what we're doing below because it's not immediately obvious what it is about.

Comment on lines 168 to 173
let rparen =
SimpleTokenizer::starts_at(value.as_ref().end(), checker.locator().contents())
.find(|token| {
token.kind == SimpleTokenKind::RParen && token.start() < call.end()
});

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if we have a.__sub__(((((b)))))?

You can use call.arguments.start (or end) to get the parentheses offsets of the call expression and retain those.

@@ -156,8 +157,45 @@ pub(crate) fn unnecessary_dunder_call(checker: &mut Checker, call: &ast::ExprCal
call.range(),
);

if let Some(fixed) = fixed {
diagnostic.set_fix(Fix::safe_edit(Edit::range_replacement(fixed, call.range())));
if let Some(mut fixed) = fixed {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the fix produces invalid code if you have

4.__sub__(
    3 
    + 
    4
)

What i understand is that we fix this to

a -
    3
    +
    4

which Python doesn't like very much

if is_in_unary {
// find the first ")" before our dunder method
let rparen =
SimpleTokenizer::starts_at(value.as_ref().end(), checker.locator().contents())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we start after the attribute to reduce the text that need to be searched?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't start after the value in this case.

We're looking for this,

(-a).__sub__(1)
###^

and the difference between

(-a).__sub__(1) and -a.__sub__(1)

It could possibly be optimized another way though, perhaps by checking the tokens between value and the first Dot 🤔

@diceroll123
Copy link
Contributor Author

diceroll123 commented Jan 27, 2024

Made some tweaks!

First and foremost: the fix is marked as unsafe. This could be conditional with more type logic in a future iteration, for sure.

The arguments within the dunder method call will now stay in parentheses if it's anything more complex than a literal/name/attribute. I'm sure that logic may be fine to expand into other expression types.

So, this example:

print(a.__sub__(
    3
    +
    4
))

now turns into

print(a - (3
    +
    4
))

And since you explicitly asked,

print(a.__sub__(((((2 - 1))))))

is now reduced to

print(a - (2 - 1))  # PLC2801

Semantically it's the same, regardless of extra parens. UP034 and/or the formatter could take care of those extra parens if we did want to keep them though...


Looking forward to opinions on improving it further! 😄

let value_slice = checker.locator().slice(value.as_ref());
let arg_slice = checker.locator().slice(arg);

if arg.is_attribute_expr() || arg.is_name_expr() || arg.is_literal_expr() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can exclude some more expressions here

  • Calls
  • Unary expressions
  • Lambda
  • If expressions (I think)?
  • Dict / Set / List / Tuple and all comprehension variants (everything with parentheses)
  • Generators
  • Subscripts
  • Starred (I think)
  • Slices?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworked it, added all of the above except Unary (because that's what got us this PR in the first place hah)

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the late review. The uv release took a lot of attention recently.

This is an excellent improvement. I think there are a few more situations where adding the parentheses isn't strictly necessary. You might want to move the logic into its own function to avoid coding it twice.

Copy link
Member

@MichaReiser MichaReiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@MichaReiser MichaReiser merged commit 8dde81a into astral-sh:main Mar 4, 2024
17 checks passed
nkxxll pushed a commit to nkxxll/ruff that referenced this pull request Mar 10, 2024
## Summary

Closes astral-sh#9572

Don't go easy on this review!

## Test Plan

`cargo test`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fixes Related to suggested fixes for violations
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug in order of operations (PLC2801)
3 participants