Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

A scala 2.13 Vector can be constructed with an invalid internal state (regression from 2.12.8) #11636

Closed
oxbowlakes opened this issue Jul 16, 2019 · 5 comments
Closed

A scala 2.13 Vector can be constructed with an invalid internal state (regression from 2.12.8) #11636

oxbowlakes opened this issue Jul 16, 2019 · 5 comments
Labels
blocker has PR library:collections regression
Milestone
2.13.2

Comments

@oxbowlakes
Copy link

I have managed to construct a Vector in an application which is in an invalid state. It has taken me some time to reproduce this, because the resultant vector cannot be iterated over, serialized, and my IDE cannot inspect it. The following code works in 2.12.8:

Welcome to Scala 2.12.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_181).
Type in expressions for evaluation. Or try :help.

scala> :paste
// Entering paste mode (ctrl-D to finish)

  val a = "O" +: Iterator.continually("E").take(2101).foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val b = "O" +: Iterator.continually("E").take(223) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val c = "O" +: Iterator.continually("E").take(135) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val d = "O" +: Iterator.continually("E").take(0)   .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val e = "O" +: Iterator.continually("E").take(376) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val f = "O" +: Iterator.continually("E").take(365) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"

  val v = a ++ b ++ c ++ d ++ e ++ f

// Exiting paste mode, now interpreting.

a: scala.collection.immutable.Vector[String] = Vector(O, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E,...

scala> println( v.map(_ => 0).size )
3212

However, in scala 2.13.0:

Welcome to Scala 2.13.0 (OpenJDK 64-Bit Server VM, Java 1.8.0_181).
Type in expressions for evaluation. Or try :help.

scala> :paste
// Entering paste mode (ctrl-D to finish)

  val a = "O" +: Iterator.continually("E").take(2101).foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val b = "O" +: Iterator.continually("E").take(223) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val c = "O" +: Iterator.continually("E").take(135) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val d = "O" +: Iterator.continually("E").take(0)   .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val e = "O" +: Iterator.continually("E").take(376) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"
  val f = "O" +: Iterator.continually("E").take(365) .foldLeft(Vector.empty[String])((v, e) => v :+ e) :+ "C"

  val v = a ++ b ++ c ++ d ++ e ++ f

// Exiting paste mode, now interpreting.

a: scala.collection.immutable.Vector[String] = Vector(O, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E, E,...

scala> println( v.map(_ => 0).size )
java.lang.NullPointerException
  at scala.collection.immutable.VectorIterator.next(Vector.scala:801)
  at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
  at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
  at scala.collection.immutable.Vector.map(Vector.scala:113)
  ... 28 elided
@SethTisue SethTisue added this to the 2.13.1 milestone Jul 16, 2019
@mpilquist
Copy link

Might be interesting -- breaks at 2466 elements.

scala> (a ++ b ++ c).map(_ => 0).size
res5: Int = 2465

scala> (a ++ b ++ c ++ d).map(_ => 0).size
java.lang.NullPointerException
  at scala.collection.immutable.VectorIterator.next(Vector.scala:801)
  at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
  at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
  at scala.collection.immutable.Vector.map(Vector.scala:113)
  ... 28 elided

scala> d.size
res7: Int = 2

scala> (a ++ b ++ c ++ Vector(1, 2)).map(_ => 0).size
java.lang.NullPointerException
  at scala.collection.immutable.VectorIterator.next(Vector.scala:801)
  at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
  at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
  at scala.collection.immutable.Vector.map(Vector.scala:113)
  ... 28 elided

scala> (a ++ b ++ c ++ Vector(1)).map(_ => 0).size
java.lang.NullPointerException
  at scala.collection.immutable.VectorIterator.next(Vector.scala:801)
  at scala.collection.StrictOptimizedIterableOps.map(StrictOptimizedIterableOps.scala:100)
  at scala.collection.StrictOptimizedIterableOps.map$(StrictOptimizedIterableOps.scala:87)
  at scala.collection.immutable.Vector.map(Vector.scala:113)
  ... 28 elided

scala> (a ++ b ++ c ++ Vector.empty).map(_ => 0).size
res10: Int = 2465

@lrytz
Copy link
Member

lrytz commented Jul 17, 2019 •
edited

Thanks for the minimal/self-contained reproducer!

@Jin-H Jin-H mentioned this issue Jul 18, 2019
Closed
@oxbowlakes
Copy link
Author

Could you opine on whether this bug could leave a Vector in a state where your code might silently do the wrong thing? A particular concern would be one which dropped elements either upon insertion or iteration.

If this is the case, it's potentially extremely serious and might warrant a community notification

szeiger added a commit to szeiger/scala that referenced this issue Sep 10, 2019
@szeiger
Revert "VectorBuilder avoids copying data if possible"
9d0f058 
This reverts commit 18224fd.

# Conflicts:
#	src/library/scala/collection/immutable/Vector.scala

Reverts scala#7588 which caused
scala/bug#11636.
@szeiger szeiger mentioned this issue Sep 10, 2019
Merged
szeiger added a commit to szeiger/scala that referenced this issue Sep 10, 2019
@szeiger
Revert "VectorBuilder avoids copying data if possible"
83bd8ae 
This reverts commit 18224fd.

# Conflicts:
#	src/library/scala/collection/immutable/Vector.scala

Reverts most of  scala#7588 which caused
scala/bug#11636.
szeiger added a commit to szeiger/scala that referenced this issue Sep 11, 2019
@szeiger
Revert "VectorBuilder avoids copying data if possible"
d4d59b1 
This reverts commit 18224fd.

# Conflicts:
#	src/library/scala/collection/immutable/Vector.scala

Reverts most of  scala#7588 which caused
scala/bug#11636.
szeiger added a commit to szeiger/scala that referenced this issue Sep 11, 2019
@szeiger
Revert "VectorBuilder avoids copying data if possible"
a5f2ce4 
This reverts commit 18224fd.

# Conflicts:
#	src/library/scala/collection/immutable/Vector.scala

Reverts most of  scala#7588 which caused
scala/bug#11636.
szeiger added a commit to szeiger/scala that referenced this issue Sep 11, 2019
@szeiger
Revert "VectorBuilder avoids copying data if possible"
a26c917 
This reverts commit 18224fd.

# Conflicts:
#	src/library/scala/collection/immutable/Vector.scala

Reverts most of  scala#7588 which caused
scala/bug#11636.
@szeiger
Copy link
Member

szeiger commented Sep 12, 2019

Fixed by scala/scala#8410 for now, including a regression test for when we try to add the optimization again.

@szeiger szeiger closed this as completed Sep 12, 2019
@szeiger szeiger mentioned this issue Nov 4, 2019
Closed
@SethTisue SethTisue modified the milestones: 2.13.1, 2.13.2 Mar 18, 2020
@SethTisue
Copy link
Member

obviated by scala/scala#8534

@oxbowlakes oxbowlakes mentioned this issue Jun 4, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
blocker has PR library:collections regression
Projects
None yet
Milestone
2.13.2
Development

Successfully merging a pull request may close this issue.

Construct Vector without an invalid internal state
5 participants
@szeiger @mpilquist @lrytz @SethTisue @oxbowlakes