Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GODRIVER-2810 Switch to polling monitoring when running within a FaaS environment #1376

Merged
merged 33 commits into from
Nov 1, 2023

Conversation

prestonvasquez
Copy link
Collaborator

GODRIVER-2810

Summary

Revert SDAM monitoring process to use the legacy polling mechanism when a FaaS runtime environment is detected.

Background & Motivation

The streaming protocol used by SDAM monitoring when communicating with MongoDB 4.4+ clusters can result in socket timeouts when the underlying runtime environment resumes after being frozen/paused then (ex: Lambda).

@prestonvasquez prestonvasquez temporarily deployed to api-report September 6, 2023 22:18 — with GitHub Actions Inactive
@github-actions
Copy link

github-actions bot commented Sep 7, 2023

API Change Report

./mongo/options

compatible changes

(*ClientOptions).SetServerMonitoringMode: added
ClientOptions.ServerMonitoringMode: added
ServerMonitoringModeAuto: added
ServerMonitoringModePoll: added
ServerMonitoringModeStream: added

./x/mongo/driver/connstring

compatible changes

ConnString.ServerMonitoringMode: added
IsValidServerMonitoringMode: added
ServerMonitoringModeAuto: added
ServerMonitoringModePoll: added
ServerMonitoringModeStream: added

@prestonvasquez prestonvasquez temporarily deployed to api-report September 7, 2023 21:33 — with GitHub Actions Inactive
@@ -125,99 +124,6 @@ func (h *Hello) Result(addr address.Address) description.Server {
return description.NewServer(addr, bson.Raw(h.res))
}

const (
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved all of this logic to the driverutil internal package so it can be shared across multiple x packages without duplication.

@prestonvasquez prestonvasquez temporarily deployed to api-report September 8, 2023 22:25 — with GitHub Actions Inactive
matthewdale
matthewdale previously approved these changes Oct 16, 2023
Copy link
Collaborator

@matthewdale matthewdale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

qingyang-hu
qingyang-hu previously approved these changes Oct 17, 2023
Copy link
Collaborator

@qingyang-hu qingyang-hu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

return events[0], events[:1], nil
}

func verifySDAMEvents(client *clientEntity, expectedEvents *expectedEvents) error {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qingyang-hu @matthewdale The addition of the 1.17 schema in this ticket and the UTR updates missed in the implementation of mongodb/specifications#1291 require this verification function.

mongo/integration/client_test.go Outdated Show resolved Hide resolved
x/mongo/driver/topology/server.go Outdated Show resolved Hide resolved
@@ -206,6 +226,7 @@ type ClientOptions struct {
RetryReads *bool
RetryWrites *bool
ServerAPIOptions *ServerAPIOptions
ServerMonitoringMode *string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We define the expected values of ServerMonitoringMode, so we should define a string value for "not set" instead of using a pointer. I recommend defining empty string as "not set".

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also define expected values for the read/write concerns. Not using a pointer here deviates from the existing pattern, which seems to intend that data in an options struct be nullable. We don't expect users to construct this data at the struct level:

Each option can be set through setter functions.

Would you mind expanding on this? And what is meant by "we should define a string value for 'not set'"? If we do go the non-pointer route, I don't think we should extend the "ServerMonitoringMode*" constants to include a "ServerMonitoringModeNotSet" case, it's not defined in the specifications and I'm not sure how it would help a user.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mostly reacting to the additional logic for handling pointers, which is unnecessary if the zero value can represent "not set". However, you're correct that the strong existing pattern in ClientOptions is to use pointers and nil for "not set". This PR doesn't seem like the right place to change that pattern. Consider my original comment resolved.

matthewdale
matthewdale previously approved these changes Oct 21, 2023
Copy link
Collaborator

@matthewdale matthewdale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! 👍

@@ -206,6 +226,7 @@ type ClientOptions struct {
RetryReads *bool
RetryWrites *bool
ServerAPIOptions *ServerAPIOptions
ServerMonitoringMode *string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was mostly reacting to the additional logic for handling pointers, which is unnecessary if the zero value can represent "not set". However, you're correct that the strong existing pattern in ClientOptions is to use pointers and nil for "not set". This PR doesn't seem like the right place to change that pattern. Consider my original comment resolved.

qingyang-hu
qingyang-hu previously approved these changes Oct 23, 2023
Copy link
Collaborator

@qingyang-hu qingyang-hu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@@ -28,6 +28,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \
tzdata \
gpg \
apt-utils \
libc6-dev \
gcc \
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C compiler is required to run a race test on non-darwin, see requirements: https://go.dev/doc/articles/race_detector#Requirements

// connMu guards connecting and disconnecting. This is necessary since
// disconnecting will await the cancelation of a started connection. The
// use case for rttMonitor.connect needs to be goroutine safe.
connMu sync.Mutex
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR requires that the rttMonitor's connection be established in a separate thread than the one server.Disconnect() is called in:
concurrency (2)

Because of this, it's possible that a disconnect could occur before a connection actually takes place, P_0. This would make any call to the shared wait group unsafe. connMu guards this case.

There should be no locking contention with this solution, the P_0 case is extremely difficult to force and there should only be one call to connect per server connection.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qingyang-hu @matthewdale I just caught a race condition in the logic. Hopefully this is the last review required

Copy link
Collaborator

@matthewdale matthewdale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good 👍

@prestonvasquez prestonvasquez merged commit 99bdb94 into mongodb:v1 Nov 1, 2023
36 checks passed
@prestonvasquez prestonvasquez deleted the GODRIVER-2810 branch November 1, 2023 19:11
blink1073 pushed a commit to blink1073/mongo-go-driver that referenced this pull request Nov 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants