Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BREAKING CHANGE: awaitable queues #1641

Merged
merged 6 commits into from May 20, 2019
Merged

BREAKING CHANGE: awaitable queues #1641

merged 6 commits into from May 20, 2019

Conversation

aearly
Copy link
Collaborator

@aearly aearly commented May 19, 2019

Closes #1586

This changes the queue API a bit, allowing queue events to be awaited, as well as the results for an individual task. You can now do:

var q = async.q(async fn (task) { /* ... */ }, 10)

await q.push(task) // awaits until task has been processed
await Promise.all(q.push([1, 2, 3])) // awaits for all tasks to be processed

q.push(5) //set and forget, this won't reject if an error happens

await queue.drain() // probably the most useful thing to come out of this change

@aearly aearly added this to the 3.0 milestone May 19, 2019
@aearly aearly added the feature label May 19, 2019
}

function on (event, handler) {
events[event].push(handler)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's quite an interesting way of handling both promises and the current API. Is the promise based API useful for any of the events besides drain?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's most useful for drain, but i can see utility coming from unsaturated, and making error reject. I'm also happy making the interface uniform for all the event-style methods.

@aearly aearly merged commit e044664 into master May 20, 2019
@darksabrefr
Copy link

await q.push(task) is not safe for use for the moment because the promise returned by q.push never resolve nor reject if the queue worker throws an error. I understand that await q.push(task) cannot throw an error because of the backward compatibility of this method and its optional callback parameter (the push and forget feature). A q.asyncPush(task) method that reject its promise shouldn't be feasible ?

You can show the problem with this code (in node or browser without the require):

const async = require('async');
let pushResult;

(async () => {
	const q = async.queue(async (task) => {
		throw new Error('Bad thing');
	});

	try {
		pushResult = q.push({some: 'data'});
		await pushResult;
		console.log('Error not catched');
	} catch (e) {
		console.log('Error catched');
	}
})();

setTimeout(() => {
	console.log('exit');
	console.log(pushResult);
}, 1000);

@aearly aearly deleted the awaitable-queues branch July 17, 2019 00:26
mdebarros added a commit to mdebarros/central-services-stream that referenced this pull request Sep 16, 2019
- Fixes for latest version of async which introduced breaking change --> caolan/async#1641
- Removed ignore dep from .ncurc.json as all issues have been resolved
- Following dependencies were updated:
 async          2.6.2  →   3.1.0
 node-rdkafka   2.6.1  →   2.7.1
 raw-body       2.4.0  →   2.4.1
 sinon          7.2.4  →   7.4.2
 tap-xunit      2.3.0  →   2.4.1
 tape          4.10.1  →  4.11.0
mdebarros added a commit to mojaloop/central-services-stream that referenced this pull request Sep 16, 2019
### Dependency updates:
- Fixes for the latest version of async which introduced a breaking change --> caolan/async#1641
- Removed ignore dependency list from .ncurc.json as all issues have been resolved
- Following dependencies were updated:
 async          2.6.2  →   3.1.0
 node-rdkafka   2.6.1  →   2.7.1
 raw-body       2.4.0  →   2.4.1
 sinon          7.2.4  →   7.4.2
 tap-xunit      2.3.0  →   2.4.1
 tape          4.10.1  →  4.11.0

### Maintenance updates:
- Replaced central-services-shared dep with central-service-logger
- Migrated from istanbul to nyc
- Moved unit tests to a test/unit folder to be consistent with other repos
- Added editorconfig, eslintignore, & eslintrc for consistency with other repos
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Await-able queues
3 participants