fix(PanicHandling): Add Options.PanicHandler #2033
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
This PR fixes #1883.
The issue is that when a device running badger is abruptly powered off it might cause a database corruption, and subsequently badger will continuously panic from inside a separate Goroutine when trying to open the database.
Panicking from inside a Goroutine causes all other Goroutines to die with an error code, and the main isssue is that having a defer calling
recover()
on other Goroutines cannot prevent this from happening, making it extremely difficult to recover from such a failure if the affected developer has no control over the panicking goroutine.Solution
The solution proposed here is to create all Goroutines with a special method called
DB._go()
that will recover from the panic if the user provides a PanicHandler on the Options argument.If a user decides to use this feature he should make sure not to ignore the panic, but this would give a window for developers to work around this issue by for example deleting the offending database and creating a new one, or even just reporting an error to the appropriate API so a human can be notified of this issue.
The main complexity in this PR is the issue regarding the use of closure and Goroutines inside for loops:
https://go.dev/doc/faq#closures_and_goroutines
Before this PR these issues were handled by passing the arguments to the Goroutines as in:
Due to the way the
DB._go()
function was implemented (i.e. receiving a closure as argument) it is not possible anymore to trust this technique to make sure the data is copied and not passed as a reference to the closure, so instead I am doing it like this now wherever necessary: