Christmas trie

[rameight]

Dedicated hasher/committer

The previous hasher.go, was a generic implementation which was used by both Commit and Hash. It does a
couple of things:

• Walk the un-hashed / dirty trie towards the leafs,
• Hash the tree on the return path,
• Create one new shadow-trie containing the new hashes, cached, which will replace the old root,
• Create one new collapsed-trie containing the hashing-preimages, which is incrementally used to create higher order hash values.
• Call store, if commit is being performed, to send nodes to the database, and also to the onleaf handler (for Commit)

This means that for every Hash or Commit, every processed node was duplicated twice, using copy. When I split this in two, it was possible to remove a lot of redundant allocs.

Commit

The Commit operation is basically read-only, with the exception of setting the dirty-flag to false.

• This means that we can skip the whole 'create-new-shadow-tree-for-new-root' creation, and cut allocs in half.
• We very seldon need to do any hashing in Commit (but sometimes still do), we can skip rlp-encoding most of the times.
• The database insertion path needs the size of a node for tracking purposes, but that can be estimated instead
• of calculated strictly.
• Sending items to onleaf and db has been split off into a separate thread, and instead this PR send those tasks via a channel.

This commit update the trie package to commit 5a9c96454e04673124eedf681ae3aa0a58ffc843 of go-thereum.

[trinhdn97]

We should separate the Iterator interface and implementation of the key-value DB into another PR, not to mess with the trie stuffs.
Also I haven't figured out how it's relevant to the trie package updates. Could you clarify why we need to implement the new methods of Iterator?

[rameight]

We should separate the Iterator interface and implementation of the key-value DB into another PR, not to mess with the trie stuffs.

I will do that in next PRs. Thanks

Also I haven't figured out how it's relevant to the trie package updates. Could you clarify why we need to implement the new methods of Iterator?

The changes of Iterator in kai/kaidb is irrelevant to trie changes. I just want to update the kaidb to the corresponding version which we have in Ethereum at the time of "Christmas Trie" PR was made. Next time, I'll split PRs to avoid confusion.