You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am using badger for a data-intensive process, writing and deleting up the 10 million values per write batch in a database of ~200 million key/value pairs.
I have the following code that is running a write batch:
func (b *Indexer) updateDB() {
wb := b.db.NewWriteBatch()
defer wb.Cancel()
...set and delete up to 10 million values
err = wb.Flush()
if err != nil {
log.Panicf("Error flushing writes to db %v", err)
}
}
This code runs for several minutes and most of the time everything is successful.
However, I noticed that when badger crashes, sometimes due to exceeded memory, or when the db is not closed correctly the data in the write batch is partially committed. Only some of the reads and writes make it to the database.
I would have expected that the write batch either succeeds in full or fails altogether, in an ACID style. Why is this not the case here?
Is there a way to have an ACID batch write or to recover the database, if it was closed ungracefully, to a previous consistent state?
The text was updated successfully, but these errors were encountered:
Question.
I am using badger for a data-intensive process, writing and deleting up the 10 million values per write batch in a database of ~200 million key/value pairs.
I have the following code that is running a write batch:
This code runs for several minutes and most of the time everything is successful.
However, I noticed that when badger crashes, sometimes due to exceeded memory, or when the db is not closed correctly the data in the write batch is partially committed. Only some of the reads and writes make it to the database.
I would have expected that the write batch either succeeds in full or fails altogether, in an ACID style. Why is this not the case here?
Is there a way to have an ACID batch write or to recover the database, if it was closed ungracefully, to a previous consistent state?
The text was updated successfully, but these errors were encountered: