Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DoctrineBridge] Idle connection listener for long running runtime #53214

Conversation

alli83
Copy link
Contributor

@alli83 alli83 commented Dec 26, 2023

Q A
Branch? 7.1
Bug fix? no
New feature? yes
Deprecations? no
Issues Fix #51661
License MIT

This pull request introduces a solution based on the RoadRunner bundle's Doctrine ORM/ODM middleware https://github.com/Baldinof/roadrunner-bundle/blob/3.x/src/Integration/Doctrine/DoctrineORMMiddleware.php#L22.
It checks the status of Doctrine connection, then if the connection is initialized and connected, it performs a 'ping' to check its viability. If the ping fails, it closes the connection.

linked to doctrine/DoctrineBundle#1739

@carsonbot carsonbot added this to the 6.4 milestone Dec 26, 2023
@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch 3 times, most recently from 07aa62f to ba1eefd Compare December 26, 2023 11:53
Copy link
Contributor

@alexandre-daubois alexandre-daubois left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions 🙂

@nicolas-grekas
Copy link
Member

nicolas-grekas commented Dec 26, 2023

Hi, thanks for working on this topic.
Would there be a way to put that concern in Doctrine itself, by decorating the Connection object?
I'm concerned that the current approach will trigger one DB call per HTTP request while not all HTTP entpoints might use Doctrine, and also by the fact this only works for HTTP while recovering from stalled DB connections is also a concern for CLI workers.

@OskarStark
Copy link
Contributor

Just reading the code, it adds a lot new stuff which looks like a new feature instead of a bugfix to me

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from ba1eefd to 3ec9cd7 Compare December 27, 2023 05:15
@dunglas
Copy link
Member

dunglas commented Dec 27, 2023

I agree that a decorator would be better. Also, would it be possible to catch errors and reconnect only in this case instead of triggering a ping? This would remove the performance hit in most cases.

@OskarStark the code probably needs to be simplified, but the underlying bug is annoying because it makes Symfony apps unstable when using FrankenPHP, RoadRunner (without the bundle), Swoole etc. The fix should be made at least in the current stable version.

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from 3ec9cd7 to 24fb643 Compare December 27, 2023 06:38
@alli83
Copy link
Contributor Author

alli83 commented Dec 27, 2023

@nicolas-grekas @dunglas ok I'll update it and try to decorate the Connection object

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from 24fb643 to 40d646a Compare December 27, 2023 06:44
@OskarStark
Copy link
Contributor

@OskarStark the code probably needs to be simplified, but the underlying bug is annoying because it makes Symfony apps unstable when using FrankenPHP, RoadRunner (without the bundle), Swoole etc. The fix should be made at least in the current stable version.

Thanks for the explanation Kevin 👍

Copy link
Contributor

@ostrolucky ostrolucky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like the approach, because as @nicolas-grekas mentioned, this will keep pinging the DB every request, even if normally request wouldn't need connection. Can't we make this DBAL middleware instead? Middleware would check if any query fails with "DB went away..." error, if so, reset everything and retry.

Also, bear in mind counterpart for messenger is at https://github.com/symfony/symfony/blob/412ea7afcb533b8a7a95e248d741b70fea505350/src/Symfony/Bridge/Doctrine/Messenger/DoctrinePingConnectionMiddleware.php, so this would be a good opportunity to extract common parts into bridge

@alli83
Copy link
Contributor Author

alli83 commented Jan 2, 2024

I don't like the approach, because as @nicolas-grekas mentioned, this will keep pinging the DB every request, even if normally request wouldn't need connection. Can't we make this DBAL middleware instead? Middleware would check if any query fails with "DB went away..." error, if so, reset everything and retry.

Also, bear in mind counterpart for messenger is at https://github.com/symfony/symfony/blob/412ea7afcb533b8a7a95e248d741b70fea505350/src/Symfony/Bridge/Doctrine/Messenger/DoctrinePingConnectionMiddleware.php, so this would be a good opportunity to extract common parts into bridge

Ok, I'll update then

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from 40d646a to eb35f9d Compare January 3, 2024 06:17
@alli83
Copy link
Contributor Author

alli83 commented Jan 3, 2024

@ostrolucky
Before refactoring (taking into account DoctrinePingConnectionMiddleware), I'm not entirely sure I grasp the specific middleware you're referring to. Are you referring to those related to Doctrine\DBAL\Driver
Doctrine\DBAL\Driver\Connection

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch 2 times, most recently from 27e36c1 to d8773ac Compare January 3, 2024 06:23
@ostrolucky
Copy link
Contributor

I was referring to middlewares that implement Doctrine\DBAL\Driver\Middleware. This is how Symfony does stuff like logging

@KDederichs
Copy link
Contributor

Any news on this/what's preventing it from being merged?

@alli83 alli83 force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from d8773ac to 7791eb2 Compare March 12, 2024 04:19
@nicolas-grekas nicolas-grekas modified the milestones: 6.4, 7.1 Apr 25, 2024
@nicolas-grekas nicolas-grekas force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from b16e918 to 3d2816c Compare April 25, 2024 09:20
Copy link
Member

@nicolas-grekas nicolas-grekas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I force-pushed to fix my last comments :)

@nicolas-grekas nicolas-grekas added Ready ❄️ Feature Freeze Important Pull Requests to finish before the next Symfony "feature freeze" labels Apr 25, 2024
Copy link
Member

@dunglas dunglas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@nicolas-grekas nicolas-grekas force-pushed the doctrine-brige-doctrine-connection-listener-long-running-runtime branch from 3d2816c to f7cc44e Compare April 25, 2024 09:30
@nicolas-grekas
Copy link
Member

Thank you @alli83.

<element key="time-sensitive">
<array>
<element key="0"><string>Symfony\Bridge\Doctrine\Middleware\Debug</string></element>
<element key="1"><string>Symfony\Bridge\Doctrine\Middleware\Debug</string></element>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nicolas-grekas Isn't Symfony\Bridge\Doctrine\Middleware\IdleConnection?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, PR welcome

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in 65ccca0

@fabpot fabpot mentioned this pull request May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug DoctrineBridge Ready ❄️ Feature Freeze Important Pull Requests to finish before the next Symfony "feature freeze" Status: Reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Doctrine left in non-operable state when used with Swoole/FrankenPHP