Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cargo queue type and tests #1567

Merged
merged 5 commits into from Aug 7, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
59 changes: 59 additions & 0 deletions lib/cargoQueue.js
@@ -0,0 +1,59 @@
import queue from './internal/queue';

/**
* Creates a `cargoQueue` object with the specified payload. Tasks added to the
* cargoQueue will be processed together (up to the `payload` limit) in `concurrency` parallel workers.
* If the all `workers` are in progress, the task is queued until one becomes available. Once
* a `worker` has completed some tasks, each callback of those tasks is
* called. Check out [these](https://camo.githubusercontent.com/6bbd36f4cf5b35a0f11a96dcd2e97711ffc2fb37/68747470733a2f2f662e636c6f75642e6769746875622e636f6d2f6173736574732f313637363837312f36383130382f62626330636662302d356632392d313165322d393734662d3333393763363464633835382e676966) [animations](https://camo.githubusercontent.com/f4810e00e1c5f5f8addbe3e9f49064fd5d102699/68747470733a2f2f662e636c6f75642e6769746875622e636f6d2f6173736574732f313637363837312f36383130312f38346339323036362d356632392d313165322d383134662d3964336430323431336266642e676966)
* for how `cargo` and `queue` work.
*
* While [`queue`]{@link module:ControlFlow.queue} passes only one task to one of a group of workers
* at a time, and [`cargo`]{@link module:ControlFlow.cargo} passes an array of tasks to a single worker,
* the cargoQueue passes an array of tasks to multiple parallel workers.
*
* @name cargoQueue
* @static
* @memberOf module:ControlFlow
* @method
* @see [async.queue]{@link module:ControlFlow.queue}
* @see [async.cargo]{@link module:ControlFLow.cargo}
* @category Control Flow
* @param {AsyncFunction} worker - An asynchronous function for processing an array
* of queued tasks. Invoked with `(tasks, callback)`.
* @param {number} [concurrency=1] - An `integer` for determining how many
* `worker` functions should be run in parallel. If omitted, the concurrency
* defaults to `1`. If the concurrency is `0`, an error is thrown.
* @param {number} [payload=Infinity] - An optional `integer` for determining
* how many tasks should be processed per round; if omitted, the default is
* unlimited.
* @returns {module:ControlFlow.CargoObject} A cargoQueue object to manage the tasks. Callbacks can
* attached as certain properties to listen for specific events during the
* lifecycle of the cargoQueue and inner queue.
* @example
*
* // create a cargoQueue object with payload 2 and concurrency 2
* var cargoQueue = async.cargoQueue(function(tasks, callback) {
* for (var i=0; i<tasks.length; i++) {
* console.log('hello ' + tasks[i].name);
* }
* callback();
* }, 2, 2);
*
* // add some items
* cargoQueue.push({name: 'foo'}, function(err) {
* console.log('finished processing foo');
* });
* cargoQueue.push({name: 'bar'}, function(err) {
* console.log('finished processing bar');
* });
* cargoQueue.push({name: 'baz'}, function(err) {
* console.log('finished processing baz');
* });
* cargoQueue.push({name: 'boo'}, function(err) {
* console.log('finished processing boo');
* });
*/
export default function cargo(worker, concurrency, payload) {
return queue(worker, concurrency, payload);
}
3 changes: 3 additions & 0 deletions lib/index.js
Expand Up @@ -70,6 +70,7 @@ import asyncify from './asyncify'
import auto from './auto'
import autoInject from './autoInject'
import cargo from './cargo'
import cargoQueue from './cargoQueue'
import compose from './compose'
import concat from './concat'
import concatLimit from './concatLimit'
Expand Down Expand Up @@ -147,6 +148,7 @@ export default {
auto,
autoInject,
cargo,
cargoQueue,
compose,
concat,
concatLimit,
Expand Down Expand Up @@ -251,6 +253,7 @@ export {
auto as auto,
autoInject as autoInject,
cargo as cargo,
cargoQueue as cargoQueue,
compose as compose,
concat as concat,
concatLimit as concatLimit,
Expand Down