Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rechunkRandomly sometimes emits everything in a single chunk #3195

Closed
Jasper-M opened this issue Mar 31, 2023 · 3 comments · May be fixed by #3205
Closed

rechunkRandomly sometimes emits everything in a single chunk #3195

Jasper-M opened this issue Mar 31, 2023 · 3 comments · May be fixed by #3205
Assignees
@yyy1000
Labels
bug good first issue

Comments

@Jasper-M
Copy link
Contributor

Jasper-M commented Mar 31, 2023 •
edited

I have five 1-element chunks and a maxFactor of 2.0, but with certain seeds rechunkRandomly emits all elements in a single chunk. The number of chunks in the input stream doesn't matter. If you have 10000 1-element chunks, you end up with a single 10000-element chunk.

val fiveChunks = Stream(1) ++ Stream(2) ++ Stream(3) ++ Stream(4) ++ Stream(5)

fiveChunks.rechunkRandomlyWithSeed(0.1, 2.0)(5).chunks.toList == List(Chunk(1,2,3,4,5))

https://scastie.scala-lang.org/TqE3oVzwQ2asDEdW8rf3uA

Other example of a wrong result:

scala> (Stream(1,1) ++ Stream(2,2) ++ Stream(3,3) ++ Stream(4,4) ++ Stream(5,5)).rechunkRandomlyWithSeed(0.1, 2.0)(9).chunks.toList
val res16: List[fs2.Chunk[Int]] = List(Chunk(1), Chunk(1, 2, 2, 3, 3, 4, 4, 5, 5))
@Jasper-M Jasper-M added the bug label Mar 31, 2023
@diesalbla
Copy link
Contributor

diesalbla commented Mar 31, 2023 •
edited

@Jasper-M Can you check if this bug already happened a few versions back? For instance, version 3.4.0? We recently changed this method in #3095

@Jasper-M
Copy link
Contributor Author

The bug also seems to be present in 3.4.0.
The issue seems to occur if you get nextSize == 0 a number of times in a row.

@yyy1000
Copy link
Contributor

yyy1000 commented Apr 11, 2023

Hi, I want to help with this.

@yyy1000 yyy1000 mentioned this issue Apr 12, 2023
Open
@Jasper-M Jasper-M mentioned this issue Aug 29, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yyy1000 yyy1000

Labels
bug good first issue
Projects
Typelevel Good First Issues
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: chunk get together when random rechunk yyy1000/fs2
5 participants
@mpilquist @diesalbla @armanbilge @Jasper-M @yyy1000