You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Approach:
We want to persist monitors at some cadence, easiest thing to would be to stop persisting on every block and instead persist on every 10th/50th block.
This will cut down IO by a factor of 10/50 but doesn't solve the thundering herd problem. All monitors will rush to get persisted after the same block.
So idea is to introduce somewhat random yet deterministic distribution scheme for monitor persists.
Partition_key will be a function of (monitor, block_height).
This partitioning strategy will alleviate thundering herd issue and hot partition problem for monitor persists and we can kind of evenly distribute the IO load.
For a node with 500 channels, this should cut down IO from 250k monitor persist calls to ~5-6k persists in an 8 hour interval.
Note this will also mean that on node restarting, a monitor will be at max 50 blocks out-of-date and we will need to sync them.
Currently on every bitcoin block update we persist all channel_monitors with updated best_block.
This can be troublesome for large node operators with 1000's of channels.
It also causes a thunder herd problem (ref), and hammers the storage with many requests all at once.
The text was updated successfully, but these errors were encountered: