Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Forknet] Copy out State and FlatState instead of deleting all other columns #11280

Open
robin-near opened this issue May 10, 2024 · 0 comments · May be fixed by #11310
Open

[Forknet] Copy out State and FlatState instead of deleting all other columns #11280

robin-near opened this issue May 10, 2024 · 0 comments · May be fixed by #11310
Assignees

Comments

@robin-near
Copy link
Contributor

In the fork-network code:

    /// Deletes DB columns that are not needed in the new chain.
    fn finalize(&self, near_config: &mut NearConfig, home_dir: &Path) -> anyhow::Result<()> {
        // Open storage with migration
        let storage = open_storage(&home_dir, near_config).unwrap();
        let store = storage.get_hot_store();

        tracing::info!("Delete unneeded columns in the original DB");
        let mut update = store.store_update();
        for col in DBCol::iter() {
            match col {
                DBCol::DbVersion | DBCol::Misc | DBCol::State | DBCol::FlatState => {}
                _ => update.delete_all(col),
            }
        }
        update.commit()?;
        Ok(())
    }

Deleting the columns probably do not save a ton of space, or at least would need a lengthy compaction step. Instead, we can do the following:

  • Copy DbVersion, Misc, and FlatState to a separate database
  • Copy only the values in State (not the nodes) to this separate database
  • Swap the new database in place.

With new research, this process should take about only 2 minutes to complete, and should leave a database that is quite tiny in size (30GB ish?)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant