New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: whenCurrentJobsFinished shouldn't initialize bclient #1347
Conversation
// Since connections are lazily initialized, we can't check queue.client | ||
// without initializing a connection. So expose a boolean we can safely | ||
// query. | ||
queue[type + 'Initialized'] = true; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couldn't you store this information in a separate data structure? furthermore, what other queue types would benefit from storing this data?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Couldn't you store this information in a separate data structure?
Not sure I follow. Do you mean you want a dedicated object on the queue to save this? Something like
queue._clientStatuses = {}; // in Queue constructor
queue._clientStatuses[type] = 'initialized'; // after initialization
Or are you thinking there's a more complex state machine that should be used?
what other queue types would benefit from storing this data?
Do you mean Redis connection types? There may be instances where client
and eclient
are also initialized unnecessarily, but I haven't tracked them down yet. That's also blocked by #1344 which means that client
is always initialized.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@manast Have you had a chance to look at this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, sorry. I meant a separate data structure.
The failing test appears to be an existing race condition: Lines 82 to 117 in c0ee6be
|
1 similar comment
This is awesome, thanks for fixing this @gabegorelick! |
This is the same change as OptimalBits#1347 but for `pause(true, true)`.
* docs: fix typos * test: fix mocha options Having `test/test_*` in `mocha.opts` meant that it was not possible to run an individual test in isolation. This may have been caused by the recent mocha version upgrade. * fix: typo in important notes section * feat(sandbox): emulate job.progress function * test: ban use of console Logging is useful in tests to debug failures, but leaving it in leads to noise. * chore: lint JS files in root directory Not a huge deal since these files are trivial, but important if they ever grow. * fix: queue.pause(true, true) doesn't pause queue Fixes OptimalBits#1532 * chore: cleanup whenCurrentJobsFinished Cleanup and document whenCurrentJobsFinished. And since it's now official API, add some tests for it. * chore: add eslint-plugin-mocha This is useful to ensure something like `.only` isn't accidentally introduced. It also includes a bunch of other useful rules. * test: delete test_cluster.js These tests have been disabled since 9a29725 was merged in 2017 (OptimalBits#472). * chore: fix line endings .eslintrc.yml got checked in with CLRF line endings. Every other file has LF line endings, so let's make it consistent. * docs: update CHANGELOG * 3.12.0 * fix(pause): don't initialize bclient if not waiting for jobs to finish This is the same change as OptimalBits#1347 but for `pause(true, true)`. * chore: cleanup lockfiles - Delete package-lock.json. Bull currently has both yarn.lock and package-lock.json files. When both are present, TravisCI uses yarn, so in practice npm has not been used by CI. But the presence of a package-lock.json file meant that contributors were not sure which lock file, if any, needed to be updated. Yarn also logs a scary warning if it detects a package-lock.json file. - Cache yarn dependencies in TravisCI. This should make CI jobs somewhat faster. - Update yarn.lock file and ensure that it remains in sync with package.json changes by using `--frozen-lockfile`. * fix: catch errors parsing invalid progress data * docs: update CHANGELOG * 3.12.1 * fix: whenCurrentJobsFinished should wait for all jobs See https://github.com/OptimalBits/bull/pull/1542/files#r356810178 * queue.clean should clean job logs as well OptimalBits#1599 * feat: add "preventParsingData" option of job to prevent data parsing * VERY SMALL TYPO ADJUSTMENTS Was reading through docs, thought these changes might be appreciated. * ci: update changelog * ci: upgrade dependencies * 3.13.0 * Generate unique JobId for repeated jobs to prevent duplicated repeated jobs Co-authored-by: sjoseph7 <17250762+sjoseph7@users.noreply.github.com> Co-authored-by: Manuel Astudillo <manuel@optimalbits.com> Co-authored-by: Gabe Gorelick <gabegorelick@gmail.com> Co-authored-by: Abhishek Soni <abhisheksoni2720@gmail.com> Co-authored-by: Doug Ayers <4142577+douglascayers@users.noreply.github.com> Co-authored-by: Stanislav Mamontov <stanislav.mamontov@gmail.com> Co-authored-by: Tom Grossman <tomg1988@gmail.com> Co-authored-by: Siarhei Pakhuta <siarhei.pakhuta@mapbox.com> Co-authored-by: Will Dembinski <wi-ski@users.noreply.github.com>
This is the same change as OptimalBits#1347 but for `pause(true, true)`.
This is the same change as OptimalBits#1347 but for `pause(true, true)`.
Fixes #1346