StackOverflowError with Promise with scalac >= 2.13.7

[yanns]

problem

When updating an application from scala 2.13.6 to 2.13.7, we can observe new StackOverflowError like:

java.lang.StackOverflowError
	at scala.concurrent.impl.Promise$Transformation.handleFailure(Promise.scala:443)
	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:506)
<our code>
	at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
<our code>
	at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
	at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:285)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:278)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2(Promise.scala:149)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2$adapted(Promise.scala:145)
	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484)
<our code>
	at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
<our code>
	at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
	at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:285)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:278)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2(Promise.scala:149)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2$adapted(Promise.scala:145)
	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484)
<our code>
	at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
<our code>
	at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
	at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete0(Promise.scala:285)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:278)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2(Promise.scala:149)
	at scala.concurrent.impl.Promise$DefaultPromise.$anonfun$zipWith$2$adapted(Promise.scala:145)
	at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:484)

I can only provide this stack trace. The <our code> is very minimal, like a synchronous execution context.

We can also observe this behavior with scala 2.13.8
We cannot observe this behavior with scala 2.13.6

I'll change the execution context to be stack safe.

You could still find this useful. I don't know if it's a regression or not.

[yanns]

I confirm that using scala.concurrent.ExecutionContext.parasitic fixes the issue.

[yanns]

Maybe:

• we had the same issue with scala 2.13.6
• one method was added in scala 2.13.7

-> this is sufficient to reach a StackOverflowError with scala 2.13.7

[som-snytt]

The behavior change is "symmetrification of zipWith" scala/scala#9655

I need a second coffee before understanding the moving parts, but presumably whatever callbacks are moved to heap in parasitic context.

Maybe the new zipWith is less friendly for "synchronous execution context", or maybe the desired solution is to submit Batchable jobs to opportunistic context described in docs for global.

[SethTisue]

not sure if @viktorklang still wants to be summoned for this kind of thing but let's find out :-)

also @lihaoyi since that's his PR

[viktorklang]

@yanns Do you have a minimal reproducer?

[yanns]

No I don't have a reproducer.
This happens on a web application using sangria to resolve GraphQL requests.
And it happens only very sporadically.

Sangria is using Promise to find values for query paths that need a Future result.
Ex: https://github.com/sangria-graphql/sangria/blob/fc86f1364a8098318ef5745f072c85b62d65d8f2/modules/core/src/main/scala/sangria/execution/Resolver.scala#L1187

So I guess this happens only on a very specific query that may need several Promises chained together.
That's why I'm guessing about #12524 (comment)

I just hope that it's not something more important.

[viktorklang]

@yanns Thanks for the background info. May I ask which ExecutionContext you're using with this?

[yanns]

It was a synchronous one:

ExecutionContext.fromExecutor(new concurrent.Executor {
   def execute(command: Runnable) = command.run()
})

I've switched it with scala.concurrent.ExecutionContext.parasitic and that fixes this issue.

[viktorklang]

@yanns

A general purpose ExecutionContext must be asynchronous in executing any Runnable that is passed into its execute-method.
A special purpose ExecutionContext may be synchronous but must only be passed to code that is explicitly safe to be run using a synchronously executing ExecutionContext.

-- https://www.scala-lang.org/api/current/scala/concurrent/ExecutionContext.html

The correct fix is to use scala.concurrent.ExecutionContext.parasitic and you have confirmed that it solves the problem :)

This is not-a-bug :)

[viktorklang]

@SethTisue Vote to close

[yanns]

I'm also ok to close this.
I just wanted to grab some attention in case there's a more serious issue behind this.

[viktorklang]

@yanns You did the right thing. Thank you :)