Skip to content
This repository has been archived by the owner on Jul 16, 2021. It is now read-only.

Extend DetectsLostConnection messages with a config array #2530

Open
wmather opened this issue Mar 11, 2021 · 0 comments
Open

Extend DetectsLostConnection messages with a config array #2530

wmather opened this issue Mar 11, 2021 · 0 comments

Comments

@wmather
Copy link

wmather commented Mar 11, 2021

The idea:
Allow us to extend the DetectsLostConnection array of strings with an array of strings in config/database.php

The background
I'm in a situation using Google Cloud Platform with managed PostgreSQL service and when the database restarts randomly, the queue workers hang. Supervisor has no control over them. I have to manually sudo kill the queue process.

That led me to the QueueWorker being able to handle this by detecting if the database connection is lost based on the predefined strings in DetectsLostConnections. If you see that link you can see the blame history for all the additions that are very specific to whichever database and service you are using. But if the specific message for your system doesn't match, it won't detect a lost connection.

For example, in the list there is a:
SQLSTATE[HY000]: General error: 7 SSL SYSCALL error: EOF detected
line in the file, but on my stack the error is reported as
SQLSTATE[08006] [7] SSL SYSCALL error: EOF detected

The text is formatted differently so it won't match. If it was matching only SSL SYSCALL error: EOF detected, then it would detect it.

I also get:
SQLSTATE[08006] [7] SSL SYSCALL error: Success
SQLSTATE[08006] [7] FATAL: the database system is in recovery mode
SQLSTATE[08006] [7] could not connect to server: Connection refused Is the server running on host "IP_ADDRESS_REDACTED" and accepting TCP/IP connections on port 5432?

The last one used to be picked up by matching against "Connection refused", until this PR made the line more specific and now it doesn't.

It seems unscalable to keep having to update the framework for every permutation of message produced by stacks where we have no control over how the messages are presented. I see some of the PRs adding extra strings are for managed databases.

There have been 9 new strings added to the framework in the last 12 months

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant