Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MongoMetricsCommandListener is not thread safe #2401

Closed
jshields-squarespace opened this issue Jan 12, 2021 · 1 comment · Fixed by #2402
Closed

MongoMetricsCommandListener is not thread safe #2401

jshields-squarespace opened this issue Jan 12, 2021 · 1 comment · Fixed by #2402
Labels
bug A general bug
Milestone

Comments

@jshields-squarespace
Copy link
Contributor

jshields-squarespace commented Jan 12, 2021

MongoMetricsCommandListener is not thread safe because it uses a shared instance of Timer.Builder. This means that concurrent calls to commandSucceeded() or commandFailed() can generate incorrect metrics.

I've created a unit test here that reproduces the issue: jshields-squarespace@80d0fd6

This test creates 4 threads that execute different Mongo operations, then checks that the metric corresponding to each command has been incremented by 1. The test fails when the threads are run concurrently (concurrent = true), but passes when the threads are run in serial (concurrent = false).

Example concurrent output:

[mongod output] iterationsCompleted=1, insertCommandCount=1, updateCommandCount=2, deleteCommandCount=0, dropIndexesCommandCount=1

org.opentest4j.AssertionFailedError: 
Expecting:
 <2L>
to be equal to:
 <1L>
but was not.

Example serial output:

[mongod output] iterationsCompleted=100, insertCommandCount=100, updateCommandCount=100, deleteCommandCount=100, dropIndexesCommandCount=100
@jshields-squarespace
Copy link
Contributor Author

I created an example fix in PR #2402, and verified that with it my example test case passes when run concurrently.

@shakuzen shakuzen added the bug A general bug label Jan 14, 2021
@shakuzen shakuzen added this to the 1.3.17 milestone Jan 14, 2021
@shakuzen shakuzen changed the title MongoMetricsCommandListener is not thread safe MongoMetricsCommandListener is not thread safe Feb 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug A general bug
Projects
None yet
2 participants