Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ChangeStream): hasNext/next should work after resume #2333

Closed
wants to merge 22 commits into from

Conversation

emadum
Copy link
Contributor

@emadum emadum commented Apr 24, 2020

Description

NODE-2548

What changed?

  • introduced getCursor method to provide fresh cursor after resuming from an error
  • refactored next/hasNext to use getCursor
  • refactored processNewChange
    • simplified to only support callbacks thanks to maybePromise work
    • extracted recreateCursor to reduce duplication in waitForTopology / createChangeStreamCursor logic
  • renamed createChangeStreamCursor to createCursor
  • refactored ChangeStream.close to use maybePromise
  • added new Change Stream Resume Error Tests
  • fixed broken mid-close tests

Are there any files to ignore?

lib/error.js Outdated Show resolved Hide resolved
@emadum emadum marked this pull request as ready for review April 26, 2020 21:28
@emadum emadum requested a review from mbroadst April 26, 2020 21:36
test/functional/change_stream.test.js Outdated Show resolved Hide resolved
lib/change_stream.js Outdated Show resolved Hide resolved
lib/change_stream.js Show resolved Hide resolved
lib/change_stream.js Outdated Show resolved Hide resolved
lib/error.js Outdated Show resolved Hide resolved
lib/error.js Outdated Show resolved Hide resolved
lib/change_stream.js Outdated Show resolved Hide resolved
@emadum emadum requested a review from mbroadst April 29, 2020 18:39
lib/change_stream.js Show resolved Hide resolved
lib/error.js Outdated Show resolved Hide resolved
test/functional/change_stream.test.js Outdated Show resolved Hide resolved
@@ -4,7 +4,7 @@ language: node_js
branches:
only:
- master
- 3.6
- next
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidentally left this change in the previous merge from 3.6, so I reverted it here. But really we should just delete .travis.yml since we don't use Travis and therefore don't maintain this file.

@@ -786,7 +793,7 @@ describe('Change Streams', function() {
}
});

it('Should return MongoNetworkError after first retry attempt fails using promises', {
it.skip('Should return MongoNetworkError after first retry attempt fails using promises', {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These twoShould return MongoNetworkError after first retry attempt fails tests started timing out now that .hasNext resumes from errors. They just keep resuming from the network errors; none of the resume attempts fail. I'm guessing these tests are quite old and it might make more sense to remove them than to try to fix them.

@emadum emadum requested a review from mbroadst May 1, 2020 13:07
let server;
let client;
let changeStream;
const oldLabelTest = label =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I just remove these tests for the old labels?

const nextP = changeStream.next();

return changeStream.close().then(() => nextP);
const nextP = () => changeStream.next();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to change nextP into a function to prevent the .next from returning successfully before the close on the next line.

expect(err).to.not.exist;
});
},
250
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this small delay because triggering the resumable error immediately after waitForStarted causes weird behavior, and isn't particularly realistic.

@@ -2800,7 +2815,7 @@ describe('Change Streams', function() {
case 2:
// only events after this point are relevant to this test
events = [];
triggerResumableError(changeStream, () => events.push('error'));
triggerResumableError(changeStream, () => events.push('error'), 0);
Copy link
Contributor Author

@emadum emadum May 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, in this case the resumable error needs to be instant because of how the test is constructed; it has to trigger before the next change event is processed.

@emadum emadum changed the base branch from 3.5 to change-stream-close-maybepromise May 3, 2020 01:02
@emadum emadum changed the base branch from change-stream-close-maybepromise to 3.5 May 3, 2020 01:48
@emadum
Copy link
Contributor Author

emadum commented May 3, 2020

I pulled a bunch of changes from this PR that weren't relevant into #2343. I'll open a new PR to replace this one with just the relevant changes, once that one is merged.

@emadum emadum closed this May 3, 2020
@emadum
Copy link
Contributor Author

emadum commented May 8, 2020

Superseded by #2360.

@emadum emadum deleted the NODE-2548/changestream-resume-fix branch May 8, 2020 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants