Commits on Jul 12, 2022

1. bundle: split large values in bundle iterator …

   Before, we'd failed to activate a bundle with a large data.json which
   would end up being attempted to load in a single write.
   
   Now, we'll recognize if the value is larger than a threshold, and split
   it into multiple blobs, which hopefully go in with a single txn.
   
   Activating the bundle in question works, but re-activating it afterwards
   still fails in a different place: eraseBundles accumulates too many writes
   in a single txn, attempting to reset the data in store.)
   
   Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>

   

   srenatus committed Jul 12, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for 1af61b5
   • Browse repository at this point

   Copy the full SHA

   1af61b5 View commit details

   Browse the repository at this point in the history

2. bundle/store: deal with erasing large bundles …

   Before, we've accumulated too-large transactions by retrieving data keys
   according to their partitioning, and deleting them one-by-one.
   
   Now, we'll do the same thing, but in an iterator passed to Truncate, which
   deals with restarting transactions.
   
   This also means that we've got an extra call to Truncate, so one Truncate
   operation more than before. However, we've been having one Truncate per
   bundle already. To fix this, we could combine multiple storage.Iterator into
   a chaining iterator, and only do one Truncate operation, but that's a future
   optimization.
   
   Signed-off-by: Stephan Renatus <stephan.renatus@gmail.com>

   

   srenatus committed Jul 12, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for 4577fad
   • Browse repository at this point

   Copy the full SHA

   4577fad View commit details

   Browse the repository at this point in the history