Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This fixes a bug where in certain circumstances greenlets can get blocked indefinitely in pool.get() even though there are items available (or at least current_size < max_size). To trigger this bug, the following conditions must be met:
1.) There are greenlets waiting on items from the pool (pool.waiting() > 0)
2.) pool.create() must throw an exception
3.) pool.get() must not be called again (or more accurately, greenlets will block until pool.get() is called again)
It seems like those would be fairly rare circumstance, especially 3.), but our use of pool triggers this any time we have a database failover. We maintain two different pool's corresponding to database servers db1 an db2. Normally, we would just use the pool of connections to db1. Suppose that db1 goes completely down (no longer responds to network requests):
1.) We'll almost certainly have pool.waiting() > 0 because queries that were taking milliseconds, now must wait on a timeout in the seconds range
2.) pool.create() will throw an exception (some kind of connect timeout)
3.) As soon as we get a network exception, we failover and start using db2's pool and don't call get() on db1's pool again.
So basically, any failover scenario cause a greenlets to block indefinitely.
This fix has two caveats:
1.) You can no longer have a pool of None objects since they now have special meaning. This was already the case for db_pool
2.) If there are greenlet's waiting and create() throws an exception we lose exception info because put() causes a greenlet switch. We raise the exception again, but it now appears to originate in pool itself.