refactor(NODE-6090): Implement CSOT logic for server selection and connection checkout

[W-A-James]

Description

What is changing?

New Error

• Added MongoOperationTimeoutError class that is thrown when a CSOT timeout is encountered

Changes to Timeout

• Add Timeout.throwIfExpired() method
• Add Timeout.remainingTime getter method

Updates to AbstractOperation

• Add timeout field
  • timeout is set at construction if the timeoutMS option is provided

Implementing CSOT behaviour for server selection

• Update Topology.selectServer to accept a timeout option which it will use determine whether it has timed out when defined. Otherwise, constructs a Timeout using the serverSelectionMS option as before
• Update Topology.selectServer to throw a MongoOperationTimeoutError on timeout when options.timeout is provided and retain previous error behaviour otherwise.
• Update Topology._connect to pass down timeout to Server.command call used to execute ping on first connection

Implementing CSOT behaviour for connection checkout

• Update Server.command to accept timeout option.
• Update ConnectionPool.checkOut to accept timeout option
  • only uses the passed in timeout if the configured serverSelectionTimeoutMS is greater than the duration on the timeout, otherwise, computes the time elapsed since server selection completed and creates timeout for the serverSelectionTimeoutMS deadline

Test changes

• Implement Server Selection prose tests from CSOT spec. Left out last two since they require implementation of the connection creation cancellation behaviour that was decided against in DRIVERS-2347
• Implement unit tests from CSOT spec related to server selection and connection checkout. Skipped tests with appropriate messages

Misc changes

• Update resolveOptions to handle timeoutMS option propagation
• • Add csotMin helper method that implements the CSOT min algorithm described here

Is there new documentation needed for these changes?

What is the motivation for this change?

Release Highlight

Fill in title or leave empty for no highlight

Double check the following

• Ran npm run check:lint script
• Self-review completed using the steps outlined here
• PR title follows the correct format: type(NODE-xxxx)[!]: description
  • Example: feat(NODE-1234)!: rewriting everything in coffeescript
• Changes are covered by tests
• New TODOs have a related JIRA ticket

[W-A-James]

if configured serverSelectionTimeoutMS < configured timeoutMS then: // i.e. we used serverSelectionTimeoutMS as the timeout for selectServer
	create a timeout that expires after (serverSelectionTimeoutMS - timeElapsed)
else:
	reuse timeout passed in via options

[nbbeeken]

looks pretty good

[nbbeeken]

Suggested change

	if (options.operationTimeout) {
	conn = await this.pool.checkOut({ timeout: options.operationTimeout });
	} else {
	conn = await this.pool.checkOut();
	}
	conn = await this.pool.checkOut({ timeout: options.operationTimeout });

TS supports just calling this because the timeout is optional

[aditi-khare-mongoDB]

I feel Warren's current code is easier to read (and easier for someone editing the code later to not accidentally make the code not CSOT spec-compliant) , but if we do end up going with this suggestion can we leave a clarifying comment?

[nbbeeken]

I am surprised because breaking this up into two calls to checkOut based on a condition that does matter is more to read without more meaningful context given. Whether or not timeout exists, there is no change to how checkOut is, practically, invoked because the typescript reports that field as optional.

I would actually take this further:

conn = await this.pool.checkOut(options);

Why do we need to make a new object here? passing through options should be fine right? Less branching paths the less there is to debug

[nbbeeken]

Can you elaborate on what may accidentally break the spec without a test warning us?

[nbbeeken]

This is still and equals check, sorry I think we discussed it but didn't leave a comment, if duration is the same then we'll create a new timeout when we can use the existing one.

[baileympearson]

drainWaitQueue is called when the topology is closed. Is it correct not to clear timeouts when the client is closed? I don't think so.

[baileympearson]

Or do we rely on drainWaitQueue rejecting each request in the wait queue, which would clear the timeout when the catch handler is run in selectServer?

[baileympearson]

The CSOTContext class sounds somewhat similar to the OperationContext idea that you had explored earlier on in the CSOT design. I'm not against it on principle, but I think we scrapped that idea at some point. Will look for any documents we had on why and report back.

The context would be similar to the timeout factory, not the operation context. We decided not to implement an operation context because we can pass CSOT-related data on the options objects in the driver. I'm proposing a CSOT context to encapsulate CSOT logic, which we would then pass through the driver on the options objects.

---

I do not consider this thread resolved. I think the current implementation is more complicated than necessary because we only sometimes re-use the timeout.

• This means server selection and connection checkout must be responsible for determining when to reuse the timeout.
• We can't unconditionally clear the timeout because it may be re-used later.

Instead, I propose we either:

1. Always create a new timeout for server selection and connection checkout.
2. Always reuse the same Timeout object, but just reset the Timeout's interval for server selection and connection checkout.

Regardless of which is chosen, I think the resultant code is simpler because server selection and connection checkout 1) do not worry about whether or not they need to use a cached timeout or create a new one 2) they can always clear the timeout.

This works especially nicely with the TimeoutFactory or a TimeoutContext, because we can encapsulate all timeout related logic into a single place that's easily unit testable. I'm partial to the factory approach:

class TimeoutFactory {
  private timeoutMS: number | null;
  private started = now();

  getTimeoutForServerSelection(): Timeout {
    // returns a timeout, handling CSOT vs Legacy timeout logic
  }
}

class Topology {

  selectServer(options: { ..., timeoutFactory: TimeoutFactory }) {
    ...
    const timeout = timeoutFactory.getTimeoutForServerSelection();
    try {
		....
	} finally {
      timeout.clear();
    }
  }
}

Note that with an approach like this, whether or not we reuse a timeout can easily be encapsulated into the TimeoutFactory by instantiating a timeout when the factory is constructed and returning the cached timeout where needed.

But a context class could suffice too:

class TimeoutContext {
  private timeoutMS: number | null;
  private started = now();

  getTimeoutForServerSelection(): number {}
}


class Topology {

  selectServer(options: { ..., timeoutContext: TimeoutContext }) {
    ...
    const timeout = Timeout.expires(timeoutFactory.getTimeoutForServerSelection());
    try {
		....
	} finally {
      timeout.clear();
    }
  }
}

An approach like this consolidates CSOT logic and can be reused outside of the main code path (i.e., topology connect).

I don't think this work needs to block this PR. But I do want to make sure we discuss this, and I'd like to consider one of these approaches in a future ticket.

[baileympearson]

Is this correct if we're auto-connecting the client?