Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

async function and await expression! #287

Closed
mysticatea opened this issue Jul 29, 2016 · 49 comments
Closed

async function and await expression! #287

mysticatea opened this issue Jul 29, 2016 · 49 comments

Comments

@mysticatea
Copy link
Member

Async functions are now stage 4 and will be included in ES2017!
https://twitter.com/bterlson/status/758826160700530688

We can support those now!

@xjamundx
Copy link
Contributor

I wonder if acorn will do it natively, or if we'll need their plugin?
On Thu, Jul 28, 2016 at 6:00 PM Toru Nagashima notifications@github.com
wrote:

Async functions are now stage 4 and will be included in ES2017!
https://twitter.com/bterlson/status/758826160700530688

We can support those now!


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#287, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAPBfyxASeFLnTXwvNhYpF5leNEW4ojDks5qaVDGgaJpZM4JXyKS
.

@mysticatea
Copy link
Member Author

I think @nzakas is waiting for response: acornjs/acorn#441

@nzakas
Copy link
Member

nzakas commented Jul 29, 2016

Yup, waiting to hear back from Acorn. If they don't want to do it yet, then we'll hack it in here.

@kaicataldo
Copy link
Member

Any progress on this? Could we start by using this plugin?

@mysticatea
Copy link
Member Author

I'm working on acornjs/acorn#446 😉

@xjamundx
Copy link
Contributor

YEAAAAH!

@kaicataldo
Copy link
Member

Wow, awesome! 🎉

@benmosher
Copy link

Released a few hours ago, looks like: acornjs/acorn#446 (comment)

@nzakas
Copy link
Member

nzakas commented Sep 7, 2016

Cool, if someone wants to put a PR together, we can merge into Espree and it will make it into the next version of ESLint.

@kaicataldo
Copy link
Member

Cool - I'll try to get to it tonight if someone doesn't beat me to it

@xjamundx
Copy link
Contributor

xjamundx commented Sep 7, 2016

If you can't get to it, I'll be backup :)

@kaicataldo
Copy link
Member

Unfortunately, didn't end up having as much time as I thought I would tonight - feel free to take it, @xjamundx!

@nzakas
Copy link
Member

nzakas commented Sep 8, 2016

I'll do it.

@nzakas
Copy link
Member

nzakas commented Sep 8, 2016

Ugh, something is really wrong - a bunch of tests are failing with parsing errors. I don't have the energy to investigate but I've pushed the issue287 branch to the repo if someone else wants to take a look.

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

K will try to take a look in a bit, thanks for starting this friends!

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

Confirmed 24 errors with the upgrade. Mostly related to the exponential operator.

@kaicataldo
Copy link
Member

Wonder if it has something to do with this:

A number of internal method signatures changed, so plugins might need
to be updated.

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

Oh n/m I didn't npm i. There are actually 96 errors.

@nzakas
Copy link
Member

nzakas commented Sep 8, 2016

Heh, I was just going to say, I got over 90 errors. :)

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

I'm looking into this change and other changes to the recoverable erorrs:
acornjs/acorn@28cbcbc

This is a pretty common error amongst the 96 (74 instances):

     SyntaxError: Unexpected token :
      at Parser.instance.raise.instance.raiseRecoverable (espree.js:9:14226)
      at Parser.instance.unexpected (espree.js:9:15709)
      at Parser.pp$3.parsePropertyValue (node_modules/acorn/dist/acorn.js:2140:12)
      at Parser.instance.parseObj (espree.js:9:13497)
      at Parser.pp$3.parseExprAtom (node_modules/acorn/dist/acorn.js:1930:19)
      at Parser.pp$3.parseExprSubscripts (node_modules/acorn/dist/acorn.js:1817:21)
      at Parser.pp$3.parseMaybeUnary (node_modules/acorn/dist/acorn.js:1794:19)
      at Parser.pp$3.parseExprOps (node_modules/acorn/dist/acorn.js:1736:21)
      at Parser.pp$3.parseMaybeConditional (node_modules/acorn/dist/acorn.js:1719:21)
      at Parser.pp$3.parseMaybeAssign (node_modules/acorn/dist/acorn.js:1696:21)
      at Parser.pp$1.parseExport (node_modules/acorn/dist/acorn.js:1220:23)
      at Parser.pp$1.parseStatement (node_modules/acorn/dist/acorn.js:752:71)
      at Parser.pp$1.parseTopLevel (node_modules/acorn/dist/acorn.js:656:25)
      at Parser.parseTopLevel (espree.js:9:8854)

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

Okay no progress yet. Back to work for a bit, will revisit later today if I can.

FYI easiest way to test one of the issues is to run:

require('./').parse(require('fs').readFileSync('./tests/fixtures/ecma-version/6/restParams-and-arrowFunctions/destructured-arrow-object.src.js', 'utf8'), { sourceType: 'module', ecmaVersion: 7 })

Funny of course that this works just fine:

require('acorn').parse(require('fs').readFileSync('./tests/fixtures/ecma-version/6/restParams-and-arrowFunctions/destructured-arrow-object.src.js', 'utf8'), { sourceType: 'module', ecmaVersion: 7 })

Also confirmed error exists in Acorn 4.0.1 and 4.0.0 but not 3.3.0.

Possibly related commit: acornjs/acorn@7c75f92

Looking again..

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

okay progress. there's a new isAsync boolean you have to pass to the parsePropertyValue function. in case you wanted to add a new boolean as the 3rd argument of a 5 argument function please read: https://ariya.io/2011/08/hall-of-api-shame-boolean-trap ;-)

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

Okay now there's only 4-5 tests failing. Of course async doesn't actually work yet, but it's a start :)

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

Current errors are these. Wondering if we can just go along with their new line numbers or if we're doing something wrong... will take a look later tonight.


  1) ecmaVersion Scripts with 6/destructuring-and-spread/error-complex-destructured-spread-first should parse correctly when sourceType is script:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:69:24)

  2) ecmaVersion Scripts with 6/destructuring-and-spread/invalid-not-final-array-empty should parse correctly when sourceType is script:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:69:24)

  3) ecmaVersion Scripts with 6/destructuring-and-spread/not-final-array should parse correctly when sourceType is script:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:69:24)

  4) ecmaVersion Modules with 6/destructuring-and-spread/error-complex-destructured-spread-first should parse correctly when sourceType is module:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:94:24)

  5) ecmaVersion Modules with 6/destructuring-and-spread/invalid-not-final-array-empty should parse correctly when sourceType is module:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:94:24)

  6) ecmaVersion Modules with 6/destructuring-and-spread/not-final-array should parse correctly when sourceType is module:

      AssertionError: expected { Object (index, lineNumber, ...) } to deeply equal { Object (index, lineNumber, ...) }
      + expected - actual

       {
      -  "column": 6
      -  "index": 5
      +  "column": 5
      +  "index": 4
         "lineNumber": 1
         "message": "Comma is not permitted after the rest element"
       }

      at Function.assert.deepEqual (node_modules/chai/lib/chai/interface/assert.js:205:32)
      at Object.module.exports.assertMatches (tests/lib/tester.js:61:16)
      at Context.<anonymous> (tests/lib/ecma-version.js:94:24)

@nzakas
Copy link
Member

nzakas commented Sep 8, 2016

We should see if the new locations are more accurate than the old. If not, maybe there's an Acorn bug.

@xjamundx
Copy link
Contributor

xjamundx commented Sep 8, 2016

I'm pretty sure the new numbers are more accurate, I'll take them for now.

@xjamundx
Copy link
Contributor

xjamundx commented Sep 9, 2016

Started adding tests and a basic implementation. Most of the tests aren't working yet and I need to be done for the night, so anyone can take over if they want or I'll try to take it back tommorow :)

@mysticatea
Copy link
Member Author

Thank you, everyone.

For reference, there are many test cases in acorn/test/tests-asyncawait.js and acorn/test/tests-trailing-commas-in-func.js

@xjamundx
Copy link
Contributor

xjamundx commented Sep 9, 2016

Great thanks @mysticatea we'll use those test cases!

@xjamundx
Copy link
Contributor

Still working on creating those tests. I remember there was some easy way to generate the tests, but I can't remember how.

@xjamundx
Copy link
Contributor

K continuing this work today...

@nzakas
Copy link
Member

nzakas commented Sep 13, 2016

Looks like my comment didn't go through :-/ Look at tools/create-tests.js

@xjamundx
Copy link
Contributor

tools/create-tests.js is working great, mainly having an issue with coming up for names for all 100+ tests I'm copying over from acorn :) Hope to have all of the test files generated by this afternoon, though not all of the cases are fully working.

@xjamundx
Copy link
Contributor

xjamundx commented Sep 14, 2016

In case it needs to be used later, I'm going to paste in my test cases (stolen from acorn):

/*!espree-section: async*/
async function foo() {}
/*!espree-section: no-async*/
function foo() {}
/*!espree-section: async-and-no-async*/
async
function foo() {}
/*!espree-section: export-async*/
export async function foo() { }
/*!espree-section: export-default-async*/
export default async function() { }
/*!espree-section: invalid-async-generator*/
async function* foo() { }
/*!espree-section: invalid-nested-async*/
async function wrap() {
    async function await() { }
}
/*!espree-section: invalid-async-generator*/
async function* foo() { }
/*!espree-section: invalid-await-param*/
async function foo(await) { }
/*!espree-section: invalid-await-identifier*/
async function foo() { return {await} }
/*!espree-section: invalid-await-identifier*/
async function foo() { return {await} }
/*!espree-section: no-async-expression*/
(function foo() { })
/*!espree-section: async-expression*/
(async function foo() { })
/*!espree-section: invalid-async-expression*/
(async
function foo() { })
/*!espree-section: invalid-async-expression-generators*/
(async function* foo() { })
/*!espree-section: export-default-async-expression*/
export default (async function() { })
/*!espree-section: export-default-async-expression*/
export default (async function() { })
/*!espree-section: arrow-func*/
a => a
/*!espree-section: arrow-func-parens*/
(a) => a
/*!espree-section: async-arrow-func*/
async a => a
/*!espree-section: async-arrow-func-parens*/
async (a) => a
/*!espree-section: async-arrow-func-parens-multi*/
async (a, b) => a
/*!espree-section: async-arrow-func-destructed*/
async ({a = b}) => a
/*!espree-section: async-arrow-func-destructed-defaults*/
async ({a: b = c}) => a
/*!espree-section: async-destructured-assignment*/
async ({a: b = c})
/*!espree-section: async-then-arrow*/
async
a => a
/*!espree-section: async-func*/
async (await)
/*!espree-section: async-yield*/
async yield => 1
/*!espree-section: object-method*/
({foo() { }})
/*!espree-section: async-object-method*/
({async foo() { }})
/*!espree-section: async-named-object-method*/
({async() { }})
/*!espree-section: async-await-named-object-method*/
({async await() { }})
/*!espree-section: class-method*/
class A {foo() { }}
/*!espree-section: async-class-method*/
class A {async foo() { }}
/*!espree-section: async-static-class-method*/
class A {static async foo() { }}
/*!espree-section: async-named-class-method*/
class A {async() { }}
/*!espree-section: static-async-named-class-method*/
class A {static async() { }}
/*!espree-section: async-named-generator-method*/
class A {*async() { }}
/*!espree-section: async-named-static-generator-method*/
class A {static* async() { }}
/*!espree-section: async-class-method-named-await*/
class A {async await() { }}
/*!espree-section: async-static-class-method-named-await*/
class A {static async await() { }}
/*!espree-section: plain-await*/
await
/*!espree-section: async-await*/
async function foo(a, b) { await a }
/*!espree-section: async-arrow-func-parens*/
(async function foo(a) { await a })
/*!espree-section: aasync-await-arrow-expression*/
(async (a) => await a)
/*!espree-section: async-await-object-method*/
({async foo(a) { await a }})
/*!espree-section: async-await-expression-class-method*/
(class {async foo(a) { await a }})
/*!espree-section: async-await-math*/
async function foo(a, b) { await a + await b }
/*!espree-section: await-identifier-math*/
function foo() { await + 1 }
/*!espree-section: async-await-identifier-math*/
async function foo() { await + 1 }
/*!espree-section: async-await-function-param*/
async function foo(a = async function foo() { await b }) {}
/*!espree-section: async-await-arrow-param*/
async function foo(a = async () => await b) {}
/*!espree-section: async-await-object-method-param*/
async function foo(a = {async bar() { await b }}) {}
/*!espree-section: async-await-class-method-param*/
async function foo(a = class {async bar() { await b }}) {}
/*!espree-section: async-await-inside-parens*/
async function wrap() {
    (a = await b)
}
/*!espree-section: async-await-destructured-default*/
async function wrap() {
    ({a = await b} = obj)
}
/*!espree-section: generator-async-func*/
function* wrap() {
    async(a = yield b)
}

and invalid cases:

/*!espree-section: invalid-async-generator*/
async function* foo() { }
/*!espree-section: invalid-nested-async*/
async function wrap() {
  async function await() { }
}
/*!espree-section: invalid-async-await-param*/
async function foo(await) { }
/*!espree-section: invalid-async-await-identifier*/
async function foo() { return {await} }
/*!espree-section: invalid-broken-line-async*/
(async
function foo() { })
/*!espree-section: invalid-async-generator-expression*/
(async function* foo() { })
/*!espree-section: invalid-await-func-expression*/
(async function await() { })
/*!espree-section: invalid-await-param-expression*/
(async function foo(await) { })
/*!espree-section: invalid-await-identifier-expression*/
(async function foo() { return {await} })
/*!espree-section: invalid-async-object*/
async ({a = b})
/*!espree-section: invalid-broken-async-arrow-before-parens*/
async
() => a
/*!espree-section: invalid-broken-async-arrow*/
async a
=> a
/*!espree-section: invalid-broken-async-arrow-after-parens*/
async ()
=> a
/*!espree-section: invalid-await-arrow-param*/
async await => 1
/*!espree-section: invalid-await-arrow-param-parens*/
async (await) => 1
/*!espree-section: invalid-await-destructured-param*/
async ({await}) => 1
/*!espree-section: invalid-await-named-destructured-param*/
async ({a: await}) => 1
/*!espree-section: invalid-await-named-destructured-array-param*/
async ([await]) => 1
/*!espree-section: invalid-broken-async-object-method*/
({async
foo() { }})
/*!espree-section: invalid-async-getter-method*/
({async get foo() { }})
/*!espree-section: invalid-async-getter-method-with-params*/
({async set foo(value) { }})
/*!espree-section: invalid-async-generator-method*/
({async* foo() { }})
/*!espree-section: invalid-nested-async-await-method*/
async function wrap() {
  ({async await() { }})
}
/*!espree-section: invalid-async-method-expression*/
({async foo() { var await }})
/*!espree-section: invalid-async-method-await-param*/
({async foo(await) { }})
/*!espree-section: invalid-async-method-return-await*/
({async foo() { return {await} }})
/*!espree-section: invalid-async-prop*/
({async foo: 1})
/*!espree-section: invalid-async-class-broken*/
class A {async
  foo() { }}
/*!espree-section: invalid-async-static-method-broken*/
class A {static async
  foo() { }}
/*!espree-section: invalid-async-constructor*/
class A {async constructor() { }}
/*!espree-section: invalid-async-class-getter*/
class A {async get foo() { }}
/*!espree-section: invalid-async-class-setter*/
class A {async set foo(value) { }}
/*!espree-section: invalid-async-class-generator*/
class A {async* foo() { }}
/*!espree-section: invalid-static-async-getter*/
class A {static async get foo() { }}
/*!espree-section: invalid-static-async-setter*/
class A {static async set foo(value) { }}
/*!espree-section: invalid-static-async-generator*/
class A {static async* foo() { }}
/*!espree-section: invalid-async-wrapped-class-await*/
async function wrap() {
  class A {async await() { }}
}
/*!espree-section: invalid-async-method-await-identifier*/
class A {async foo() { var await }}
/*!espree-section: invalid-async-method-await-param*/
class A {async foo(await) { }}
/*!espree-section: invalid-async-class-method-await*/
class A {async foo() { return {await} }}
/*!espree-section: invalid-plain-await*/
await
/*!espree-section: invalid-await-top-level*/
await a
/*!espree-section: invalid-async-plain-await*/
async function foo() { await }
/*!espree-section: invalid-async-expression-plain-await*/
(async function foo() { await })
/*!espree-section: invalid-plain-await-arrow*/
async () => await
/*!espree-section: invalid-async-method-empty-await*/
({async foo() { await }})
/*!espree-section: invalid-async-class-method-empty-await*/
(class {async foo() { await }})
/*!espree-section: invalid-async-generator*/
async function foo(a = await b) {}
/*!espree-section: invalid-async-generator*/
(async function foo(a = await b) {})
/*!espree-section: invalid-async-generator*/
async (a = await b) => {}
/*!espree-section: invalid-async-generator*/
async function wrapper() {
  async (a = await b) => {}
}
/*!espree-section: invalid-async-generator*/
({async foo(a = await b) {}})
/*!espree-section: invalid-async-generator*/
(class {async foo(a = await b) {}})
/*!espree-section: invalid-class-await-extend*/
async function foo(a = class extends (await b) {}) {}
/*!espree-section: invalid-await-default*/
async function wrap() {
  (a = await b) => a
}
/*!espree-section: invalid-await-destructured-default*/
async function wrap() {
  ({a = await b} = obj) => a
}
/*!espree-section: invalid-yield-default*/
function* wrap() {
  async(a = yield b) => a
}

@xjamundx
Copy link
Contributor

Updated the branch with fairly working code, realized I need to update the create-test.js to include ecmaVersion: 8, so running it again and hopefully won't get 200 errors :)

@xjamundx
Copy link
Contributor

Issue appears to be that the newly generated nodes are missing start and end properties in some cases. Probably doing something silly.

@nzakas
Copy link
Member

nzakas commented Sep 14, 2016

I'm not sure if this is the issue, but in the test fixtures we don't include start and end, and the output from Espree is normalized to skip those in the tests.

@xjamundx
Copy link
Contributor

@nzakas cool that makes sense I made some minor modifications to create-test, so maybe I turned them back on. I'm hoping I'll get this fixed pretty soon. It's going well-ish :)

@xjamundx
Copy link
Contributor

xjamundx commented Sep 14, 2016

@nzakas it seems like create-test is adding start and end. Is that something I did wrong specifically in the implementation of async/await or do i just need to manually remove them?

The branch is up to date if you want to pull it down and see.

@xjamundx
Copy link
Contributor

added this thing to create-test and now more things are passing:

 recursivelyRemoveStartAndEnd(result)
    function recursivelyRemoveStartAndEnd(o) {
        if (Array.isArray(o)) {
            o.forEach(recursivelyRemoveStartAndEnd)
        } else if (o && typeof o === 'object') {
            delete o.start
            delete o.end
            Object.keys(o).filter(key => key !== 'loc').forEach((key) => {
                recursivelyRemoveStartAndEnd(o[key])
            })
        }
    }

i'm obviously doing something wrong :-p

@xjamundx
Copy link
Contributor

Once this was fixed new issues revolve around the await identifier being banned in strict mode. Need to tweak how we put tests for strict/module mode. I'm soooo rusty, but I think I'll get it.

@nzakas
Copy link
Member

nzakas commented Sep 14, 2016

@xjamundx Espree outputs start and end by default, it comes from Acorn and is hard to remove. So create-test.js is just using the raw output from Espree. You can use this to easily strip stuff you don't want in the output:
https://github.com/eslint/espree/blob/master/tests/lib/tester.js#L26

@xjamundx
Copy link
Contributor

Thanks @nzakas !

@xjamundx
Copy link
Contributor

Found what I need. Filename magic :)

var scriptOnlyTestFiles = allTestFiles.filter(function(filename) {
    return filename.indexOf("modules") === -1;
});

var moduleTestFiles = allTestFiles.filter(function(filename) {
    return filename.indexOf("not-strict") === -1 && filename.indexOf("edge-cases") === -1;
});

@xjamundx
Copy link
Contributor

Got all tests to pass. Updating now. Probably some additional tests may be needed, but it's getting close hopefully!

@xjamundx
Copy link
Contributor

Let me open a PR

@xjamundx
Copy link
Contributor

Note to self:
next time don't try to add so many tests :-p

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants