Implement support for async generator functions and for-await statements

[zenparsing]

This PR depends on babel/babylon@b926e40. Tests will fail until the next babylon release, but submitting early for feedback.

This change implements the async iteration proposal, currently at stage 2. It includes the following features:

• Transforms async generator functions (async function* g() { }) to wrapped generator functions, similar to the current async-to-generator transform.
• Transforms for-await statements into for loops containing yield expressions.

Thanks!

Implementation Notes

Additional comments inline.

Async Generator Functions

Async generator functions are transformed into regular generator functions, wrapped with a call to a new helper function (asyncGenerator.wrap). Within the async generator function, await expressions are translated into yield expressions, where the argument is "boxed" using another helper function (asyncGenerator.await). The wrapper function detects instances of the "boxed" values and knows that it should "pump" the generator with the awaited value instead of yielding to the consumer.

Rather than duplicate the async wrapping transform in babel-helper-remap-async-to-generator, I chose to refactor that helper to allow an optional await wrapping callee.

for-await

Since for-await statements can also appear inside of regular async functions and must take into account whether or not await expressions are "boxed", the for-await translation is done inside babel-helper-remap-async-to-generator. Most of the for-await transform logic comes from the existing for-of transformer.

[zenparsing]

I looks to me like the current async-to-generator transform is unnecessarily transforming arrows that are nested within nested function declarations and expressions.

Is there some reason why arrows nested within a nested function would need to be translated?

[jprichardson]

Since this functionality was in Babel v5, any idea when this will make it into the current Babel?

[hzoo]

Made the babylon release https://github.com/babel/babylon/releases/tag/v6.8.0

[codecov-io]

Current coverage is 87.84%

Merging #3473 into master will increase coverage by +<.01%

1. 5 files (not in diff) in packages were modified. more
   • Misses +2
   • Hits +3
2. 3 files (not in diff) in packages were created. more

@@             master      #3473   diff @@
==========================================
  Files           194        197     +3   
  Lines         10119      10173    +54   
  Methods        1535       1544     +9   
  Messages          0          0          
  Branches       2245       2253     +8   
==========================================
+ Hits           8886       8936    +50   
- Misses         1233       1237     +4   
  Partials          0          0          

Powered by Codecov. Last updated by 2607f35...f19c10d

[zenparsing]

Can someone help explain the failing "codecov/changes" check? I can quite figure out what's going on there.

[hzoo]

I think there are some issues with codecov (recently), I wouldn't worry about it atm

[zenparsing]

OK. It seems like we should have some eval tests, but I'm not sure if that's really possible given the async nature of this transform? Are there any functional tests for async functions?

[hzoo]

I don't think there have been - only https://github.com/babel/babel/blob/master/packages/babel-plugin-transform-async-to-generator/test/fixtures/regression/T6882/exec.js which was a test for hoisting http://phabricator.babeljs.io/T6882. Otherwise I would check out the existing exec.js tests

[jprichardson]

Anymore progress on this? This unlocks a lot of potential and code broke since it was in Babel v5 :(

[baslr]

Then, finally we can iterate nicely over a database query cursor

[jprichardson]

Is there anything I can help with here to get this integrated into a Babel release?

[zenparsing]

@jprichardson There a failing code coverage test which I can't make heads or tails of, and there really ought to be some sort of "exec" tests, but I haven't had time to sit down and figure out how to make that work for async operations.

[hzoo]

@zenparsing Don't worry about the coverage issue like I mentioned above

[ghost]

Just found out that Babel does NOT transpile arr.includes to ES5 arr.indexOf.

[loganfsmyth]

@wifiextender Correct, you load babel-polyfill for that. This has nothing to do with this thread though.

[tonyxiao]

According to transform-regenerator docs we already support async generators? https://github.com/babel/babel/blob/master/packages/babel-plugin-transform-regenerator/README.md

[Jessidhia]

@tonyxiao that is if you use the regenerator runtime. This seems to be for use with native generators.

[tonyxiao]

I see. @Kovensky got it thanks.

[alexeyraspopov]

@zenparsing, do you still have plans to finish this? It will be awesome!

[jprichardson]

Isn't this now possible with this https://github.com/babel/babel/tree/master/packages/babel-plugin-syntax-async-generators ?

[hzoo]

Yes but we need a code review and some more exec tests as mentioned above - unless we want users to be our testers 😛

[domenic]

FYI, we're hoping to push the async iteration proposal to stage 3 at the next TC39 meeting (in two weeks' time). I totally understand if the Babel maintainers don't have the time to take this pull request over the finish line, but it would be ideal if Babel supported such a stage 3 feature.

[hzoo]

I'l try to take a look this week or at least finish up the tests so we can get this used/tested

[hzoo]

@zenparsing can you fixup the merge conflicts? If I don't get around to the tests we'll just merge as is so we can get something out there like @domenic said

[hzoo]

Going to merge manually in another pr

[hzoo]

Made #4576, feel free to comment there. (fixed merge conflicts, added async exec test)

[hzoo]

Thanks @zenparsing, sorry for the super long delay