Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Floor for NewVectorIterator.slice limit #10518

Merged
merged 1 commit into from Sep 26, 2023

Conversation

kapunga
Copy link
Contributor

@kapunga kapunga commented Aug 28, 2023

Fixes scala/bug#12823
Adds a floor of 0 to the limit parameter of NewVectorIterator.slice. This matches the base behavior of Iterator.slice and Vector.slice.

@scala-jenkins scala-jenkins added this to the 2.13.13 milestone Aug 28, 2023
@kapunga kapunga force-pushed the 12823-fix-vector-slice-overflow branch 2 times, most recently from 4ece6e3 to 7b6cb90 Compare August 28, 2023 06:07
@SethTisue SethTisue added the library:collections PRs involving changes to the standard collection library label Aug 28, 2023
@SethTisue SethTisue requested a review from a team August 28, 2023 14:03
Copy link
Contributor

@ansvonwa ansvonwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me and seems pretty solid against me playing around with it.
Two things I noted:

  • The infix max isn't used anywhere else in Vector.scala, so using the imported mmax (mmax(until, 0)) may be more consistent. (And maybe save one allocation? not sure..)
  • The slice method leaves the remaining iterator in a different state than Iterator.slice does. But as the Documentation of Iterator clearly states "one should never use an iterator after calling a method on it" this seems fine.

@Ichoran
Copy link
Contributor

Ichoran commented Aug 28, 2023

It looks fine to me. I would tend to favor more direct logic, more like:

val n =
  if (until <= 0 || from >= until) 0
  else if (from > 0) {
    drop(from)
    until - from
  }
  else until
take(n)

but the current version seems to do the same thing, just in a slightly harder-to-parse way (and doing a bit more computation in the from > until case).

take(_until)
_until - from
} else _until
take(n)
Copy link
Contributor

@He-Pin He-Pin Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (until<= 0 || until<= from)  Iterator.empty else {
 if (from > 0) drop(from)
 take(if(from < 0) until else until- from)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unsure about your comment. Is this a third proposal? If so, I think it should be until instead of util, and there are some formatting and correctness issues (for instance, .slice(0,0) returns the entire vector, so this would need to be e.g. Iterator.empty).

Copy link
Contributor

@He-Pin He-Pin Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ansvonwa yes, I edit it on my phone, thanks for point out, but I think you can update your pr to avoid the usage of max. and @Ichoran's proposal is very direct too.

@kapunga
Copy link
Contributor Author

kapunga commented Aug 31, 2023

  • The infix max isn't used anywhere else in Vector.scala, so using the imported mmax (mmax(until, 0)) may be more consistent. (And maybe save one allocation? not sure..)

Makes sense to keep things consistent. I'll change this.

  • The slice method leaves the remaining iterator in a different state than Iterator.slice does. But as the Documentation of Iterator clearly states "one should never use an iterator after calling a method on it" this seems fine.

I agree that it's fine. The original PR for this file clearly had a lot of thought put into performance, I'm hesitant to make changes for changes sake and limited this PR to just fixing the immediate bug.

if(from > 0) {
drop(from)
until - from
Copy link
Contributor

@He-Pin He-Pin Aug 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the most quick fix can be mmax(util, 0), inline it here, otherwise lgtm.

@ansvonwa
Copy link
Contributor

I'm hesitant to make changes for changes sake and limited this PR to just fixing the immediate bug.

Definitely a good habit.
For completeness: The different state only occurs in out of bounds cases, namely from > until. There, your solution still performs the drop while @Ichoran's and the default Iterator don't. Bus as said, this should be fine.

I wonder how much would break if each such operation on an Iterator would set it to a "dirty" state where every future operation throws 😈

@SethTisue
Copy link
Member

SethTisue commented Sep 19, 2023

@kapunga Looks like this is ready for merge — but before we do that, can you squash the two commits into one? (We don't squash-merge in this repo.)

Fixes scala/bug#12823
Adds a floor of 0 to the limit parameter of NewVectorIterator.slice.
This matches the base behavior of Iterator.slice and Vector.slice.
@kapunga kapunga force-pushed the 12823-fix-vector-slice-overflow branch from 50320c4 to b860d66 Compare September 26, 2023 00:03
@kapunga
Copy link
Contributor Author

kapunga commented Sep 26, 2023

@kapunga Looks like this is ready for merge — but before we do that, can you squash the two commits into one? (We don't squash-merge in this repo.)

@SethTisue done!

@SethTisue SethTisue merged commit 59b2ede into scala:2.13.x Sep 26, 2023
3 checks passed
@SethTisue
Copy link
Member

thank you @kapunga and reviewers!

@som-snytt
Copy link
Contributor

The merge has a red ex ❌ presumably unrelated to the PR.

 [error] Test scala.collection.IteratorTest.flatMap is memory efficient in previous element failed: org.junit.ComparisonFailure: expected:<[first]> but was:<[third]>, took 0.011 sec
[error]     at scala.collection.IteratorTest.flatMap is memory efficient in previous element(IteratorTest.scala:842)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]     at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[error]     at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]     at java.lang.reflect.Method.invoke(Method.java:568)
[error]     ...

It looks like the test relies on a weak reference and is prone to flake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
library:collections PRs involving changes to the standard collection library
Projects
None yet
7 participants