Compiler support for scala-async, with a state machine transform phase running after erasure [ci: last-only]

[retronym]

scala-async (SIP-22) is currently implemented as a macro.

This pull request incorporates the bulk of the async transform into a fully blown compiler phase of the standard compiler. scala-async would remain as a simple macro that expands into the skeleton state machine class including adaptors for scala.concurrent.Future.

Other front end macros could also target the async phase adapting to other awaitable data types. The test cases include an integration with java.util.concurrent.CompletableFuture. It should be also be straightforward to integrate with monix.Task and the like now, unlike before.

Why abandon the macro approach?

• Typer is a somewhat unnatural place in the compiler pipeline for the expansion. The ANF transform is markedly simpler once things after uncurry and easier again after erasure. Custom integrations of async that use async pattern extactors and guards are only possible when running after patmat.
• Async translated blocks generate larger ASTs than the original source code. Such a a translation is better deferred to later in the pipeline for performance reasons.
• Async leans heavily on compiler APIs, and some of these are unavailable int the macro API facade.

Integrate Async into the compiler

• Import the scala-async code base into scala.tools.nsc.transform.async
• The front end macro is required to expand to async(expr)(execCtx) into (approximately) {val temp = execCtx; class stateMachine { def apply(tr: Try) = { expr; () } }; new stateMachine.start()}. Wrapping expr in a class lets intervening compiler phases do their job (e.g. captured enclosing values give rise to outer pointer usage).
• The async phase proper runs after erasure. As before, it consists of a) a selective ANF transform of the expression; b) a translation of control flow that crosses async boundaries into separate states of a state machine; c) lifting locals that are referred to from multiple states into members of the state machine.
• Refactor to be idiomatic compiler phase (use normal Transformers/Traversers APIs, use other internal APIs rather than c.internal_.
• This functionality is opt-in with -Xasync. A new version of scala-async (prototype) will be needed to take advantage.
• Test cases have been imported into test/async and junit/test/scala/tools/nsc/transform/async.

Improve the implementation of the ANF transform

• Refactor the code for better performance and maintainability
• Be more selective about when to transform code.
• Simplify with the assumption that only post-erasure Tree shapes will be seen.

Improve the implementation of the State Machine Transform

• Refactor the code for better performance and maintainability
• Drastically reduce the number of states needed to represent pattern matches and if/else constructs with "sparse" usages of await among the branches. Blocks prior to async boundaries are now kept int the same state as the If or LabelDef itself.
• Remove the restriction about using await in the operand of Boolean#{||, &&} and instead rewrite these to If trees.

Improve performance

• Compile time performance is improved markedly. This is due to emitting fewer states, performing less bookkeeping, and avoiding or fixing inefficiencies in the way we interact with compiler facilities like tree attachments, tree transformers.
• Runtime performance is likely improved for certain use cases, although the savings may be negligible compared to the cost of async boundaries for many realistic workloads. For each state transition that we avoid, we save one tableswitch, some null field assignments and read/writes of the state variable. Another benefit would be needing to lift fewer values into fields (saving space in the state machine and the put/get field cost.

TODO

• Review/Refine the names of the methods in AsyncStateMachine
• Longer term plan for how to avoid the need for the heuristic approach to deferring @compileTimeOnly await methods
• real world testing against code bases already usnig scala-async
• test coverage for:
  • error cases in UnsupportedAwaitAnalyzer
  • if (tree.tpe =:= definitions.NothingTpe) { branch in transformMatchOrIf
  • case td@TypeDef(_, _, tparams, rhs) => in Lifter (maybe dead code)
  • ThicketTransformer (add a unit test)
  • case Apply(fun, arg1 :: arg2 :: Nil) if isBooleanShortCircuit(fun.symbol) => in AsyncTraverser
• Fix positions of synthetic trees, the position of the apply method (the start of the async block or method) isn't suitable! Either use NoPosition of the preciding position.
• Test and update IntelliJ's debugger support for the changes to mangled names: Debugger improvements including for Scala Async retronym/intellij-scala#1

[sjrd]

Can these changes create new shapes of trees using LabelDefs, notably in the interaction with pattern matching? That would potentially mean more work on Scala.js' side.

[retronym]

@sjrd Yes, I'm using label jumps like continue

def apply(tr: Try[AnyRef]) = {
:while1
while (true) {
  state match {
     case <i> => 
        ...
          GOTO while1() // continue a new iteration of the dispatch loop
        ...
          GOTO case12() // forward jump to a `caseN` or `matchEnd` label that may
        ...
     case <j> =>
         :case12
     case _ => throw ...
  }
  GOTO while1()
}

The GOTO while1 would need to be translated to a Javascript continue.

The GOTO matchEnd or GOTO caseNext labels now can target a label that is not in the same Block, which I expect would be problematic for your translation. These could be replaced with an extra pass (either by the async phase or by Scala.js) with state = <state containing label>; while1().

I was also experimenting with a novel use of a LabelDef to encapsulate a few instructions used before each async boundary. Results: https://gist.github.com/retronym/c219f8759bd92b8f65804fbaeffcb6ec. That change isn't included in this PR though.

[sjrd]

We don't compile to JavaScript, but to our IR. Our IR doesn't have continue, but it has something much more fun: labeled blocks (see section 3.2.6 of my thesis). From what you're showing, it seems we'll definitely need some more special casing to recognize that shape, but that we can translate it to the following IR:

while (true) {
  while1[void]: {
    val (newState, awaitable) = await[(int, Awaitable)]: {
      state match {
        case <i> =>
          ...
          return@while1 void
          ...
          return@await (j, future)
        case _ => ...
      }
      return@while1 void
    } // end await
    state = newState
    if (isComplete(awaitable))
      tr = awaitable.get
    else
      {awaitable.onComplete(this); return}
  } // end while1
}

[sjrd]

It seems I replied to an earlier version of your comment.

For case labels, we already have a solution, as long as their shape doesn't get more complicated because of the transform.

[sjrd]

I'd like to discuss how we can make this testable in Scala.js. At the moment it seems there are essentially two kinds of tests:

• partest tests that actually delegate to a kind of JUnit runner, with tests that systematically perform Await.result
• "driven" tests relying on a JUnit test that instantiates a compiler, then runs the compiled code via reflection

Neither kind will be testable in Scala.js, which becomes problematic now that the support of async can generate new shapes of trees. It means that we'll never be able to reliably test any of this in the context of Scala.js.

Could we instead use one way to write async tests that is (or can be made) compatible with Scala.js? Off the top of my head, I imagine we could have a single AsyncTest helper in partest-extras, that would like:

abstract class AsyncTest {
  def test: Future[Any]

  def main(args: Array[String]): Unit {
    Await.result(test, Duration.Inf)
  }
}

Then, all tests that need running would be written as partests that inherit from this class. In Scala.js, we would override that single class so that it doesn't use Await.result, but instead delegates to the event loop. That would be manageable, and would allow all these tests to be testable also on Scala.js.

[sjrd]

With a few more changes on Scala.js' side, I could make all the tests in edge-cases pass :)

I noticed that, with the latest state of this PR, (almost) all generated switch'es have a last state that is empty dead code, and that would loop forever if reached. For example, the very first testBasic() in smoketest.scala generates:

  final class Test$stateMachine$async$1 extends scala.tools.partest.async.OptionStateMachine {
    <synthetic> <stable> private[this] var await$1: Object = _;
    override def apply(tr$async: Option): Unit = while$(){
      try {
        Test$stateMachine$async$1.this.state() match {
          case 0 => {
            val awaitable$async: Some = new Some(scala.Int.box(1));
            tr$async = Test$stateMachine$async$1.this.getCompleted(awaitable$async);
            Test$stateMachine$async$1.this.state_=(1);
            if (null.!=(tr$async))
              while$()
            else
              {
                Test$stateMachine$async$1.this.onComplete(awaitable$async);
                return ()
              }
          }
          case 1 => {
            {
              val tryGetResult$async: Object = Test$stateMachine$async$1.this.tryGet(tr$async);
              if (Test$stateMachine$async$1.this.==(tryGetResult$async))
                Test$stateMachine$async$1.this.await$1 = return ()
              else
                Test$stateMachine$async$1.this.await$1 = tryGetResult$async.$asInstanceOf[Object]()
            };
            val awaitable$async: Some = new Some(scala.Int.box(2));
            tr$async = Test$stateMachine$async$1.this.getCompleted(awaitable$async);
            Test$stateMachine$async$1.this.state_=(2);
            if (null.!=(tr$async))
              while$()
            else
              {
                Test$stateMachine$async$1.this.onComplete(awaitable$async);
                return ()
              }
          }
          case 2 => {
            <synthetic> val await$2: Object = {
              val tryGetResult$async: Object = Test$stateMachine$async$1.this.tryGet(tr$async);
              if (Test$stateMachine$async$1.this.==(tryGetResult$async))
                return ()
              else
                tryGetResult$async.$asInstanceOf[Object]()
            };
            <synthetic> val x$1: Int = scala.Int.unbox(await$2);
            <synthetic> val x$2: Int = scala.Int.unbox(Test$stateMachine$async$1.this.await$1).+(x$1);
            Test$stateMachine$async$1.this.completeSuccess(scala.Int.box(x$2));
            return ();
            Test$stateMachine$async$1.this.await$1 = null
          }
          case 3 => {
            ()
          }
          case _ => throw new IllegalStateException(java.lang.String.valueOf(Test$stateMachine$async$1.this.state()))
        }
      } catch {
        case (throwable$async @ (_: Throwable)) => {
          Test$stateMachine$async$1.this.completeFailure(throwable$async);
          return ()
        }
      };
      while$()
    };
    def <init>(): Test$stateMachine$async$1 = {
      Test$stateMachine$async$1.super.<init>();
      ()
    }
  }

Note the case 3 => { () }. If reached, it would loop forever since it doesn't change the state, but no other code sets the state to 3. The same happens in all async blocks I looked at.

This is not wrong/harmful; it's just useless trees.

[lrytz]

A few minor comments. I also pushed a commit to remove (hopefully) unused code.

[lrytz]

About the risk for regressions that this change may cause:

• All non-trivial changes in this PR are behind a new -Xasync compiler flag (for now)
• Existing scala-async releases will continue to work exactly the same way with compilers that include this change