"Generator functions work flow" API page not exactly correct

[alvaro-cuesta]

I came across https://github.com/iter-tools/iter-tools/wiki/Generator-functions-work-flow via bustle/streaming-iterables#22 which was a pretty interesting detail I didn't know, but the example didn't really make sense to me (the finally is going to get called anyways), so I tested the actual implementation.

Here's a counterexample:

async function* realGenerator(query) {
  try {
    console.log('open db', query)

    yield 1
    yield 2
    yield 3
  } finally {
    console.log('close db', query)
  }
}

const fakeGenerator = (query) => ({
  [Symbol.asyncIterator]() {
    return this
  },

  i: 1,

  next() {
    if (this.i === 1) {
      console.log('open db', query)
    }

    if (this.i <= 3) {
      return Promise.resolve({ value: this.i++, done: false })
    }

    return Promise.resolve({ value: undefined, done: true })
  },

  return(value) {
    console.log('close db', query)

    i = 1337

    return Promise.resolve({ value, done: true })
  },
})

async function main(g) {
  {
    console.log('Breaking early:')
    for await (const record of g('query 1')) {
      console.log('a', record)
      break
    }
  }

  {
    console.log('\nFull loop:')
    for await (const record of g('query 2')) {
      console.log('b', record)
    }
  }

  {
    console.log('\nManual looping, no return:')
    const gen = g('query 3')
    while (true) {
      const next = await gen.next()
      if (next.done) break
      console.log('c', next.value)
    }
  }

  {
    console.log('\nManual looping with return:')
    const gen = g('query 3')
    try {
      while (true) {
        const next = await gen.next()
        if (next.done) break
        console.log('c', next.value)
      }
    } finally {
      if (typeof gen.return === 'function') gen.return()
    }
  }
}

main(realGenerator)
  .then((_) => {
    console.log('\n------\n')
    main(fakeGenerator)
  })
  .then((_) => {})

This outputs:

Breaking early:
open db query 1
a 1
close db query 1

Full loop:
open db query 2
b 1
b 2
b 3
close db query 2

Manual looping, no return:
open db query 3
c 1
c 2
c 3
close db query 3

Manual looping with return:
open db query 3
c 1
c 2
c 3
close db query 3

------

Breaking early:
open db query 1
a 1
close db query 1

Full loop:
open db query 2
b 1
b 2

Manual looping, no return:
open db query 3
c 1
c 2

Manual looping with return:
open db query 3
c 1
c 2
close db query 3

Notice that:

1. On the generator with finally, "close db" is always printed
2. On the fake generator, "close db" is only printed in the breaking loop (which is kinda surprising, see below) and in the manual .return() call

Since return() is a method of Generator and not part of iterable nor iterator protocols, does it make sense to support the fake's return()? It's not part of any public interface and in fact I wouldn't expect to get my return() called if I implemented it by coincidence, but both Node and Firefox call it 😕

In the real generator the finally block is going to get called anyways. I guess the key takeaway is to call return(), but not as in the wiki example, but only if you break early, i.e. the wiki page breaking condition shouldn't be next.done, at least if we aim to conform to for ... of behavior (which I still think is pretty surprising and not mentioned anywhere in MDN... I'm searching the actual spec but it's kinda hard to find).

Anything I'm missing? What do you think?

[conartist6]

I forgot that page existed.

You are correct, return is part of the iterator spec and it will only be called on early termination. If it exists on your iterator it will be called by the language in for loops, destructuring, etc in early return situations. It is assumed that whatever cleanup return does the iterator will do itself if it terminates normally. iter-tools is similarly careful not to call return()if the iterator has already completed normally.finally` in generators does muddy the waters a bit because of course it happens both for normal and "abrupt completion" (that is the technical term you can look for in the spec). What iter-tools does then, and what you should do if you need to work with an iterator directly is:

function *doStuff(iterator) {
  let abrupt = true;
  try {
    while(!iterator.next().done) { /*...*/ }
    abrupt = false;
  } finally {
    if (abrupt) iterator.return();
  }
}

[alvaro-cuesta]

return is part of the iterator spec

Is it? It's not mentioned in MDN's iteration protocols (nor any related page). I'm trying to read ECMA-262 but I'm not getting anywhere for now.

[alvaro-cuesta]

Found it. https://exploringjs.com/es6/ch_iteration.html#sec_iteration-protocol-in-depth indeed lists return and links to the spec.

Feel free to close the issue (or leave it open if you intend to update the wiki page).

EDIT: related from the book:

return() should only be called if an iterator hasn’t be exhausted. For example, for-of calls return() whenever it is left “abruptly” (before it is finished). The following operations cause abrupt exits: break, continue (with a label of an outer block), return, throw.

There's a whole section about closing iterators. I was interested in the updated wiki page to refer colleagues to it, but that book is a must!