High amount of commit_tx calls

[TonyGiorgio]

Wanted to open up a discussion on this. I hope it's not my implementation of an a fedimint db but we store version number anytime we store KV data in indexeddb and I'm noticing a very high number of writes to disk whenever a lightning payment happens. I'm my experience, it's like ~30-40 times (I'm sure this might widely vary). I'm only actually writing to disk whenever commit_tx gets called.

I do this with Lightning data and also enforce storing remotely in the commit, and it allows us to have very high guarantees that there's no loss of data locally or remotely before proceeding with the lightning htlc dance. The lightning channel writes are incredibly efficient and kept to a minimum. However, due to the high amount of writes with fedimint, I'm unable to practically enforce saving remotely ~40 times consecutively for each lightning payment. It can take 15+ seconds to enforce remote saves.

So this leads us to currently "fire and forget" for remote fedimint storage, but all those writes still want to happen in the background, which is very inefficient (we could try to do some ~50ms delayed writer to help here), but we enforce highest number wins anyways so latest state eventually persists as long as there's no network issues. Though I would like to have less chance of losing ecash remotely. Especially when we can't get the fedimint backup solution to work in the browser yet.

Am I correct that ~40 commits happen during the lifetime of a lightning payment? Or perhaps I have a bad custom indexeddb implementation. It should work very similarly to webimint's localstorage implementation.

Is there any room for efficiency with the writes that need to be 100% necessary?

[elsirion]

Sorry to hear that, but that sounds like a number that I'd expect, if not a little low. We just didn't have any incentive to optimize this so far since DB writes are typically cheap enough to not meaningfully impact performance. I didn't anticipate there ever being an expensive DB like your remote DB.

Why there are so many commits: each state transition of a client state machine gets committed. Issuing e-cash produces about 3 state transitions iirc and currently each e-cash note is issued in its own state machine (tracked here #3741). So if your amount is represented as 10 e-cash notes you have 20 commits for issuing these plus a few more for the LN input/output state machine and some other misc stuff that gets triggered.

Short term your idea of batching writes to remote sounds as good as it will get.

[TonyGiorgio]

Thank you, I'll try that.