New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
core/types: remove support for legacy receipt/log storage encoding #22852
core/types: remove support for legacy receipt/log storage encoding #22852
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, but let's hear what karalabe thinks too
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Safe to delete as we introduced the freezer at the same time as the new format and the freezer moved everything reencoding, so nobody should have any old format stuff any more.
Note that this comment is intended only to help guide others who have the same problem, we are not expecting a fix. For anyone who sees the following in their geth console logs with v1.10.4 or later:
Or sees the following error when they attempt to export their chaindata:
What probably happened is that at some point in the past, your geth database (chaindata) got silently corrupted and failed to automatically upgrade to the modern receipt/log storage encoding. With this change (#22852) go-ethereum removes the capability to read these legacy logs. Possible solutions include: |
The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like #17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in #22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com>
The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like ethereum#17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in ethereum#22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com>
* eth/tracers: expose contextual infos (block hash, tx hash, tx index) * eth/tracers: convert int/hash values from context into js object (ethereum#23108) * Convert int/hash values from context into js object * Use js fixed buffer Co-authored-by: William <william.berman@coinbase.com> * cmd/geth: fix IPC probe in les test (ethereum#23094) Previously, the test waited a second and then failed if geth had not started. This caused the test to fail intermittently. This change checks whether the IPC is open 10 times over a 5 second period and then fails if geth is still not available. * eth/gasprice: fix typo in comment (ethereum#22998) * core/state: remove unused methods ReturnGas, GetStorageProofByHash (ethereum#23092) Co-authored-by: lidongwei <lidongwei@huobi.com> * eth/downloader: increase downloader block body allowance (ethereum#23074) This change increases the cache size from 64 to 256 Mb for block bodies. Benchmarks have shown this to be one bottleneck when trying to achieve higher download speeds. The commit also includes a minor optimization for header inserts in package core: previously, the presence of headers in the database was checked for every header before writing it. With the change, if one header fails the presence check, all subsequent headers are also assumed to be missing. This is an improvement because in practice, the headers are almost always missing during sync. * p2p/enode: fix method doc (ethereum#23115) This is an obvious spelling error Co-authored-by: liuyaxiong <liuyaxiong@inspur.com> * core, eth: fix precompile addresses for tracers (ethereum#23097) * core,eth/tracers: make isPrecompiled dependent on HF * eth/tracers: use keys when constructing chain config struct * eth/tracers: dont initialize activePrecompiles with random value * eth/gasprice: implement feeHistory API (ethereum#23033) * eth/gasprice: implement feeHistory API * eth/gasprice: factored out resolveBlockRange * eth/gasprice: add sanity check for missing block * eth/gasprice: fetch actual gas used from receipts * miner, eth/gasprice: add PendingBlockAndReceipts * internal/ethapi: use hexutil.Big * eth/gasprice: return error when requesting beyond head block * eth/gasprice: fixed tests and return errors correctly * eth/gasprice: rename receiver name * eth/gasprice: return directly if blockCount == 0 Co-authored-by: rjl493456442 <garyrong0905@gmail.com> * accounts/abi/bind: fix gas price suggestion with pre EIP-1559 clients (ethereum#23102) This fixes transaction sending in the case where an app using go-ethereum v1.10.4 is talking to a pre-EIP-1559 RPC node. In this case, the eth_maxPriorityFeePerGas endpoint is not available and we can only rely on eth_gasPrice. * p2p/server: fix method name in comment (ethereum#23123) * ethclient/gethclient: RPC client wrapper for geth-specific API (ethereum#22977) This commit adds the package gethclient which is similar to the ethclient and implements some geth specific functionality. Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com> * cmd/geth: dont fail on deprecated toml config fields (ethereum#23118) * internal/ethapi: fix panic in access list creation (ethereum#23133) Fixes test failure in the last commit. * all: removed blockhash from statedb (ethereum#23126) This PR removes the blockhash from the statedb * eth/fetcher, trie: unit test reliability fixes (ethereum#23020) Some tests take quite some time during exit, which I think causes some appveyor fails like this: https://ci.appveyor.com/project/ethereum/go-ethereum/builds/39511210/job/xhom84eg2e4uulq3 One of the things that seem to take time during exit is waiting (up to 100ms) for the syncbloom to close. This PR changes it to use a channel, instead of looping with a 100ms wait. This also includes some unrelated changes improving the reliability of eth/fetcher tests, which fail a lot because they are time-dependent. * crypto: gofuzz build directives (ethereum#23137) * eth/tracers: improve tracing performance (ethereum#23016) Improves the performance of debug.traceTransaction * go.mod: update UPNP dependency (ethereum#23116) * les: avoid shutdown hang (ethereum#23139) * consensus/clique: avoid a copy in clique (ethereum#23149) * consensus/clique:optimize to avoid a copy in clique * consensus/clique: test for sealhash Co-authored-by: Martin Holst Swende <martin@swende.se> * cmd/evm, eth/ethconfig: regenerate struct codecs (ethereum#23140) * ethstats: fix full node interface post 1559 * dockerfile: get rid of make and env, see if that fixes builds * cmd/puppeth: fix dashboard crash caused by updated base image * core: fix bad parent hash when jumping to genesis in setHead (ethereum#23162) * p2p: fix array out of bounds issue (ethereum#23165) * fuzzing: fix typo in fuzzer definitions (ethereum#23169) * eth, miner: add RPC method to modify miner gaslimit (pre london: ceiling) (ethereum#23134) * core/types: sanity check the basefee length inside a header (ethereum#23171) * all: remove noop vm config flags (ethereum#23111) * all: rm external interpreter and ewasm config * core/vm: rm Interpreter interface * cmd/geth: deprecate interpreter config fields * cmd/devp2p: fixes for eth and discv4 tests (ethereum#23155) This PR fixes a false positive PONG 'to' endpoint mismatch seen in hive tests: got {IP:172.17.0.7 UDP:44025 TCP:44025}, want {IP:172.17.0.7 UDP:44025 TCP:0} Co-authored-by: Felix Lange <fjl@twurst.com> * core/types: remove LogForStorage type (ethereum#23173) The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like ethereum#17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in ethereum#22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com> * core/types: go generate (ethereum#23177) * core, params: define london block at 12965000 (ethereum#23176) * core, params: define london block at 12965000 * core/forkid: fix test * eth/gasprice, internal/ethapi, miner: minor feehistory fixes * params: update CHTs for the 1.10.5 release * core/state: avoid unnecessary alloc in trie prefetcher (ethereum#23198) * internal/ethapi: fix transaction APIs (ethereum#23179) * internal/ethapi: fix transaction APIs * internal/ethapi: fix typo * internal/ethapi: address comments * internal/ethapi: address comment from Peter * internal: get pending and queued transaction by address (ethereum#22992) * core, eth, internal, les, light: get pending and queued transaction by address * core: tiny nitpick fixes * light: tiny nitpick Co-authored-by: mark <mark@amis.com> Co-authored-by: Péter Szilágyi <peterke@gmail.com> * consensus/clique: implement getSigner API method (ethereum#22987) * clique: implement getSignerForBlock * consensus/clique: use blockNrOrHash in getSignerForBlock * consensus/clique: implement getSigner * consensus/clique: fixed rlp decoding * consensus/clique: use Author instead of getSigner * consensus/clique: nit nit nit * consensus/clique: nit nit nit * feat: merged changes from head Co-authored-by: Péter Szilágyi <peterke@gmail.com> Co-authored-by: williamberman <wlbberman@gmail.com> Co-authored-by: William <william.berman@coinbase.com> Co-authored-by: piersy <pierspowlesland@gmail.com> Co-authored-by: haryu703 <34744512+haryu703@users.noreply.github.com> Co-authored-by: Li Dongwei <lidw1988@126.com> Co-authored-by: lidongwei <lidongwei@huobi.com> Co-authored-by: Martin Holst Swende <martin@swende.se> Co-authored-by: ForLina <471133417@qq.com> Co-authored-by: liuyaxiong <liuyaxiong@inspur.com> Co-authored-by: Sina Mahmoodi <1591639+s1na@users.noreply.github.com> Co-authored-by: Felföldi Zsolt <zsfelfoldi@gmail.com> Co-authored-by: rjl493456442 <garyrong0905@gmail.com> Co-authored-by: Pierre R <p.rousset@gmail.com> Co-authored-by: Ahyun <urbanart2251@gmail.com> Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de> Co-authored-by: Edgar Aroutiounian <edgar.factorial@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com> Co-authored-by: ucwong <ucwong@126.com> Co-authored-by: Evolution404 <35091674+Evolution404@users.noreply.github.com> Co-authored-by: Mark <markya0616@gmail.com> Co-authored-by: mark <mark@amis.com> Co-authored-by: Raymond Yeh <ray@geek.sg>
@shoenseiwaso not sure you are still around. Are you willing to recompile the geth with these diffs? I am curious why it's failed. Since we have the freezer to re-encode the legacy receipts. No legacy format receipts should be left. diff --git a/core/rawdb/accessors_chain.go b/core/rawdb/accessors_chain.go
index 76132bf37..871ab0b64 100644
--- a/core/rawdb/accessors_chain.go
+++ b/core/rawdb/accessors_chain.go
@@ -572,7 +572,7 @@ func ReadRawReceipts(db ethdb.Reader, hash common.Hash, number uint64) types.Rec
// Convert the receipts from their storage form to their internal representation
storageReceipts := []*types.ReceiptForStorage{}
if err := rlp.DecodeBytes(data, &storageReceipts); err != nil {
- log.Error("Invalid receipt array RLP", "hash", hash, "err", err)
+ log.Error("Invalid receipt array RLP", "number", number, "hash", hash.Hex(), "err", err)
return nil
}
receipts := make(types.Receipts, len(storageReceipts)) |
Let's discuss at #23245 |
…oding (ethereum#22852)" This reverts commit 643fd0e.
…thereum#22852) * core/types: remove support for legacy receipt storage encoding * core/types: remove support for legacy log storage encoding
* Revert "core/types: go generate (ethereum#23177)" This reverts commit 00b922f. * Revert "core/types: remove LogForStorage type (ethereum#23173)" This reverts commit 7522642. * Revert "core/types: remove support for legacy receipt/log storage encoding (ethereum#22852)" This reverts commit 643fd0e.
…thereum#22852) * core/types: remove support for legacy receipt storage encoding * core/types: remove support for legacy log storage encoding
The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like ethereum#17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in ethereum#22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com>
* Revert "core/types: go generate (ethereum#23177)" This reverts commit 00b922f. * Revert "core/types: remove LogForStorage type (ethereum#23173)" This reverts commit 7522642. * Revert "core/types: remove support for legacy receipt/log storage encoding (ethereum#22852)" This reverts commit 643fd0e.
The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like ethereum#17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in ethereum#22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com>
* Revert "core/types: go generate (ethereum#23177)" This reverts commit 00b922f. * Revert "core/types: remove LogForStorage type (ethereum#23173)" This reverts commit 7522642. * Revert "core/types: remove support for legacy receipt/log storage encoding (ethereum#22852)" This reverts commit 643fd0e.
…thereum#22852) * core/types: remove support for legacy receipt storage encoding * core/types: remove support for legacy log storage encoding
The encoding of Log and LogForStorage is exactly the same now. After tracking it down it seems like ethereum#17106 changed the storage schema of logs to be the same as the consensus encoding. Support for the legacy format was dropped in ethereum#22852 and if I'm not wrong there's no reason anymore to have these two equivalent types. Since the RLP encoding simply contains the first three fields of Log, we can also avoid creating a temporary struct for encoding/decoding, and use the rlp:"-" tag in Log instead. Note: this is an API change in core/types. We decided it's OK to make this change because LogForStorage is an implementation detail of go-ethereum and the type has zero uses outside of package core/types. Co-authored-by: Felix Lange <fjl@twurst.com>
* Revert "core/types: go generate (ethereum#23177)" This reverts commit 00b922f. * Revert "core/types: remove LogForStorage type (ethereum#23173)" This reverts commit 7522642. * Revert "core/types: remove support for legacy receipt/log storage encoding (ethereum#22852)" This reverts commit 643fd0e.
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt #22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted #23247. Since then we added a command to migrate legacy dbs #24028. As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt ethereum#22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted ethereum#23247. Since then we added a command to migrate legacy dbs ethereum#24028. As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
This PR drops the legacy receipt types, the freezer-migrate command and the startup check. The previous attempt ethereum#22852 at this failed because there were users who still had legacy receipts in their db, so it had to be reverted ethereum#23247. Since then we added a command to migrate legacy dbs ethereum#24028. As of the last hardforks all users either must have done the migration, or used the --ignore-legacy-receipts flag which will stop working now.
This removes the database upgrade path for receipts and logs stored before April 2019.