Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform processes input in parallel #1543

Closed
uusdfg opened this issue Jun 10, 2018 · 2 comments
Closed

transform processes input in parallel #1543

uusdfg opened this issue Jun 10, 2018 · 2 comments

Comments

@uusdfg
Copy link

uusdfg commented Jun 10, 2018

What version of async are you using?
async 2.6.1.

Which environment did the issue occur in (Node version/browser version)
Node.js v8.11.2.

What did you do? Please include a minimal reproducable case illustrating issue.

const async = require('async');

function delay(cb) {
  setTimeout(cb, Math.floor(Math.random() * 20));
}

function exerciseTransform(times, cb) {
  console.info(`Running transform ${times} times...`);
  async.times(times, (n, nextTime) =>
    async.transform([1, 2, 3, 4, 5], [], (acc, item, index, next) =>
      delay(() => { acc.push(item); next(); }), nextTime),
    (err, results) => { console.info(results); cb(); });
}

function exerciseReduce(times, cb) {
  console.info(`Running reduce ${times} times...`);
  async.times(times, (n, nextTime) =>
    async.reduce([1, 2, 3, 4, 5], [], (memo, item, next) =>
      delay(() => { memo.push(item); next(null, memo); }), nextTime),
    (err, results) => { console.info(results); cb(); });
}

async.series([
  async.apply(exerciseTransform, 5),
  async.apply(exerciseReduce, 5),
], () => {});

What did you expect to happen?
Because transform is a relative of reduce, I would expect it to process the input array serially.

What was the actual result?
Transform processes the input array in parallel.

Running transform 5 times...
[ [ 3, 1, 5, 4, 2 ],
  [ 1, 3, 4, 2, 5 ],
  [ 2, 4, 5, 3, 1 ],
  [ 1, 4, 5, 3, 2 ],
  [ 4, 5, 2, 1, 3 ] ]
Running reduce 5 times...
[ [ 1, 2, 3, 4, 5 ],
  [ 1, 2, 3, 4, 5 ],
  [ 1, 2, 3, 4, 5 ],
  [ 1, 2, 3, 4, 5 ],
  [ 1, 2, 3, 4, 5 ] ]

I'm not sure if anyone might depend on the current behavior, so maybe this is really a request for a transformSeries function.

@aearly
Copy link
Collaborator

aearly commented Jul 2, 2018

transform was created mainly as a way to process objects, where key ordering doesn't matter, and also to be the parallel cousin of reduce.

Maintaining order in the accumulator is the job of the iteratee function. Your example would work as expected if you did:

function exerciseTransform(times, cb) {
  console.info(`Running transform ${times} times...`);
  async.times(times, (n, nextTime) =>
    async.transform([1, 2, 3, 4, 5], [], (acc, item, index, next) =>
      delay(() => { acc[index] = item; next(); }), nextTime), // <--
    (err, results) => { console.info(results); cb(); });
}

@uusdfg
Copy link
Author

uusdfg commented Jul 2, 2018

I'm sorry, I didn't realize that transform was intended to run in parallel. I had expected the same left-to-right semantics as reduce.

I think it would help if the documentation for transform called out this distinction explicitly. The documentation currently says that transform, "iterates over each element in series," which suggests that it's supposed to be serial.

aearly added a commit that referenced this issue Jul 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants