Merge v1.10.7 #1695

trianglesphere · 2021-09-16T20:31:46Z

Description

This merges changes made after 1.9.25 up to 1.10.7 as well as some espresso specific HF changes.

Notable Changes from Upstream

EIP-1559 tx pool support ethereum/go-ethereum#22898. This had to be adapted to multiple fee currencies
EIP-2718: Typed Transaction Envelope + EIP-2930 ethereum/go-ethereum#21502. This modified the transaction types
all: implement eip-1559 ethereum/go-ethereum#22837. This implemented 1559. There is a celo specific implementation in Implement CIP-43: BlockContext(now rename to SysContractCallCtx) #1696
Please note that reverting to Geth v1.9.19 or prior after upgrading to v1.9.20 is not possible without a resync because the blockchain database layout has changed.

Changes from Upstream

Snap sync is completely disabled and hidden from end users
The catalyst API is not included
Lots of flags changes: See Merge upstream up to 1.10.7 #1699 (comment)

Notes for reviewers

With the new transaction types, core/types got overhauled to supported multiple types.
With 1559, the txpool was rewritten. We've placed a layer of modification over it to handle multiple fee currencies.
There were some changes in the supported p2p protocol versions + some of the files were moved around
With 1559, the blockcontext was moved & is also used in state_transition
The eth protocols got moved around (file wise).

Tested

Sync tests are being ran. It appears to sync without issue.

Related issues

Fixes Merge upstream up to 1.10.7 #1699

Backwards compatibility

This updates the database layout so while it is possible to upgrade old nodes to this version, downgrading is not supported. See https://github.com/ethereum/go-ethereum/releases/tag/v1.9.20

Please note that reverting to Geth v1.9.19 or prior after upgrading to v1.9.20 is not possible without a resync because the blockchain database layout has changed.

params: update CHTs for the 1.10.5 release

* internal/ethapi: fix transaction APIs * internal/ethapi: fix typo * internal/ethapi: address comments * internal/ethapi: address comment from Peter

* core, eth, internal, les, light: get pending and queued transaction by address * core: tiny nitpick fixes * light: tiny nitpick Co-authored-by: mark <mark@amis.com> Co-authored-by: Péter Szilágyi <peterke@gmail.com>

* clique: implement getSignerForBlock * consensus/clique: use blockNrOrHash in getSignerForBlock * consensus/clique: implement getSigner * consensus/clique: fixed rlp decoding * consensus/clique: use Author instead of getSigner * consensus/clique: nit nit nit * consensus/clique: nit nit nit

When processing a transaction with London fork rules, EIP-1559 mandates checking that the sender must have sufficient balance to cover gas * gasFeeCap. In the EIP's pseudocode, this check happens after the value transferred by the transaction has already been deducted. However, in go-ethereum, the balance has not yet been updated when the check happens, and therefore needs to be added explicitly. Co-authored-by: Martin Holst Swende <martin@swende.se>

* Revert "core/types: go generate (#23177)" This reverts commit 00b922f. * Revert "core/types: remove LogForStorage type (#23173)" This reverts commit 7522642. * Revert "core/types: remove support for legacy receipt/log storage encoding (#22852)" This reverts commit 643fd0e.

…(#23239) * internal/ethapi/api: use hexutil.uint for blockCount parameter instead of int for feeHistory * return hex value for oldestBlock instead of number * return uint64 from oracle.resolveBlockRange * eth/gasprice: fixed test Co-authored-by: Zsolt Felfoldi <zsfelfoldi@gmail.com>

* internal/ethapi: revert + fix properly in al tracer * internal/ethapi: use toMessage instead of creating new message * internal/ethapi: remove ineffassign * core: fix invalid unmarshalling, fix test Co-authored-by: Martin Holst Swende <martin@swende.se>

Ticket #23273 found a flaw where we were unable to sign legacy-transactions using the external signer, even if we're still on non-london network. That's fixed in this PR. Additionally, I found that even when supplying all parameters, it was impossible to sign a london-transaction on an unsynched node. It's a pretty common usecase that someone wants to sign a transaction using an unsynced 'vanilla' node, providing all necessary data. Our setDefaults, however, insisted on checking the current block against the config. This PR therefore adds a case, so that if both MaxPriorityFeePerGas and MaxFeePerGas are provided, we accept them as given. OBS This PR fixes a regression -- on current master, we are unable to sign a london-transaction unless the node is synched, which may break scenarios where geth (or clef) is used as a cold wallet. Fixes #23273

When the nodeCount is less than 10, it will panic with the out of bound error. How about we just skip this round, when rand1 and rand2 are equal?

This PR moves (some) account types into a standalone package, to avoid depending on signer/core from accounts/external.

The makes it such that the prefetcher associated with a stateDB object is always stopped once the state associated with it is done. Note: The prefetcher could probably be ran for a longer period of time, but there's no other clean location to stop it at while properly managing the lifetime of the object.

This commit adds and calls createConverstionFunctions to properly sort the transactions in the list by price. It first calculates the effective miner tip in the fee currency of the transaction and then converts that to CELO to globally sort all transactions before inclusion into the block.

This cleans up the tx list and the last instance of needing to convert to the currency type.

NB: Fees are for logging information, not for consensus.

mcortesi

Modules: p2p, node, graphql, event, ethdb, ethstats, ethclient, internal

Only missing eth, les, light modules

mcortesi · 2021-11-18T23:27:22Z

ethstats/ethstats.go

-		return fmt.Errorf("invalid netstats url: \"%s\", should be nodename@host:port", url)
+// parseEthstatsURL parses the netstats connection url.
+// URL argument should be of the form <name@host:port>
+func parseEthstatsURL(url string, name *string, host *string) error {


just a reminder, to test ethstats in baklava or testnet before final release @gastonponti

ethstats/ethstats.go

ethclient/ethclient.go

internal/web3ext/web3ext.go

internal/ethapi/api.go

mcortesi

Finished the review!

eth/downloader/modes.go

mcortesi · 2021-11-19T21:53:20Z

.golangci.yml

-    - staticcheck
-    - unused
+    - misspell
+    # - staticcheck


renable the ones we had before

The ones we had result in a huge number of false positives that upstream geth doesn't care about. It's a huge ongoing maintenance burden and pulls us further away from upstream.

would love to reevaluate this in a future PR. They do help finding bugs!

Dockerfile

Dockerfile.alltools

Dockerfile

Dockerfile.alltools

trianglesphere · 2021-11-30T19:22:17Z

@mcortesi direct merging seems to be disabled

mcortesi

Approved, but i'd like the comments here to be addressed, can be done on a future PR

mcortesi · 2021-12-01T22:22:35Z

accounts/abi/bind/base.go

+	// if err != nil {
+	// 	return nil, err
+	// }
+	// TODO: Use GPM here


Still missing?

mcortesi · 2021-12-01T22:24:36Z

accounts/keystore/presale.go

@@ -37,7 +37,10 @@ func importPreSaleKey(keyStore keyStore, keyJSON []byte, password string) (accou
 	if err != nil {
 		return accounts.Account{}, nil, err
 	}
-	key.Id = uuid.NewRandom()
+	key.Id, err = uuid.NewRandom()


Added an issue for later #1759

mcortesi · 2021-12-01T22:28:08Z

accounts/usbwallet/ledger.go

@@ -208,7 +215,30 @@ func (w *ledgerDriver) SignPersonalMessage(path accounts.DerivationPath, message
 	return w.ledgerSignData(path, message)
 }

-// ledgerVersion retrieves the current version of the Celo wallet app running
+/* TODO: add functionality to the Ledger's Celo app


Added issue #1760

mcortesi · 2021-12-01T22:29:39Z

cmd/faucet/faucet.go

@@ -14,7 +14,7 @@
 // You should have received a copy of the GNU General Public License
 // along with go-ethereum. If not, see <http://www.gnu.org/licenses/>.

-// faucet is a Ether faucet backed by a light client.
+// faucet is an Ether faucet backed by a light client.


mcortesi · 2021-12-01T22:31:13Z

cmd/geth/dao_test.go

@@ -108,15 +108,16 @@ func testDAOForkBlockNewChain(t *testing.T, test int, genesis string, expectBloc
 		if err := ioutil.WriteFile(json, []byte(genesis), 0600); err != nil {
 			t.Fatalf("test %d: failed to write genesis file: %v", test, err)
 		}
-		runGeth(t, "--datadir", datadir, "--nousb", "--networkid", "1337", "init", json).WaitExit()
+		runGeth(t, "--datadir", datadir, "--networkid", "1337", "init", json).WaitExit()


mcortesi · 2021-12-02T01:48:52Z

miner/stress/1559/main.go

+	espressoBlock = big.NewInt(30) // Predefined london fork block for activating eip 1559.
+)
+
+func main() {


mcortesi · 2021-12-02T01:49:40Z

tests/fuzzers/les/debug/main.go

+	"github.com/celo-org/celo-blockchain/tests/fuzzers/les"
+)
+
+func main() {


mcortesi · 2021-12-02T01:49:49Z

tests/fuzzers/rangeproof/debug/main.go

+	"github.com/celo-org/celo-blockchain/tests/fuzzers/rangeproof"
+)
+
+func main() {


mcortesi · 2021-12-02T01:49:56Z

tests/fuzzers/vflux/debug/main.go

+	"github.com/celo-org/celo-blockchain/tests/fuzzers/vflux"
+)
+
+func main() {


mcortesi · 2021-12-02T01:52:11Z

.golangci.yml

-    - staticcheck
-    - unused
+    - misspell
+    # - staticcheck


karalabe and others added 30 commits July 9, 2021 14:27

params: update CHTs for the 1.10.5 release

afd4227

Merge pull request #23183 from karalabe/cht-1.10.5

58b0420

params: update CHTs for the 1.10.5 release

core/state: avoid unnecessary alloc in trie prefetcher (#23198)

8fe47b0

internal/ethapi: fix transaction APIs (#23179)

846badc

* internal/ethapi: fix transaction APIs * internal/ethapi: fix typo * internal/ethapi: address comments * internal/ethapi: address comment from Peter

internal: get pending and queued transaction by address (#22992)

c49e065

* core, eth, internal, les, light: get pending and queued transaction by address * core: tiny nitpick fixes * light: tiny nitpick Co-authored-by: mark <mark@amis.com> Co-authored-by: Péter Szilágyi <peterke@gmail.com>

internal/web3ext: remove unused console APIs (#23208)

1fac96c

params: release Geth v1.10.5, Exodus Cluster

33ca98e

params: begin v1.10.6 release cycle

f0b1bdd

node: fix stopping websocket rpc.Server (#23211)

89fde59

eth/tracers: fix typo in test name (#23218)

a5e3aa6

les: fix eth_sendTransaction API (#23215)

f05419f

params: release go-ethereum v1.10.6 stable

576681f

params: begin v1.10.7 release cycle

a1f16bc

cmd/geth: update vulnerability testdata (#23252)

5c89ec9

accounts/abi/bind: parse ABI only once, create metadata struct (#22583)

b20bc5c

accounts/abi/bind: set Context in TransactOpts (#23188)

3aea432

consensus/ethash: less allocation during mining (#23199)

0730acc

README: fix default sync mode (#23282)

56e9001

all: change blacklist terms

523866c

tests: update tests/testdata to v9.0.4 (london) (#23279)

fb8ea59

go.mod: upgrade to goupnp v1.0.2 (#23197)

64a5e12

go.mod: bump golang.org/x/text to v0.3.6 (#23291)

b157bae

p2p/simulations: fix unlikely crash in probabilistic connect (#23200)

8f11d27

When the nodeCount is less than 10, it will panic with the out of bound error. How about we just skip this round, when rand1 and rand2 are equal?

signer/core: move API JSON types to separate package (#23275)

295bc35

This PR moves (some) account types into a standalone package, to avoid depending on signer/core from accounts/external.

Joshua Gutow added 7 commits November 17, 2021 12:47

Add documentation for priceHeap

0f94d86

core/types: Clean up comments in transaction.go

6f4607a

core/txpool: Continue to clean up currency handling

4e1761e

This cleans up the tx list and the last instance of needing to convert to the currency type.

miner: Properly calculate fees

a9243bc

miner: Check for espresso when calculating fees

c40f4d0

NB: Fees are for logging information, not for consensus.

mcortesi reviewed Nov 19, 2021

View reviewed changes

Joshua Gutow added 4 commits November 19, 2021 11:38

Implement 1559 in ethstats

fdf8789

Add tracking ticket for FeeHistory API

44e736f

Remove unimplemented method

99462ff

API: Use proper baseFee when selecting transactions

8748860

mcortesi reviewed Nov 19, 2021

View reviewed changes

Joshua Gutow added 3 commits November 19, 2021 15:47

dockerfile: Cleanup

c642468

More info on how snap sync is disabled

0acdfa5

Merge branch 'master' into trianglesphere/work

b9ee58c

gastonponti mentioned this pull request Nov 26, 2021

[CIP-50] optional replay protection #1748

Merged

hbandura reviewed Nov 30, 2021

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

hbandura reviewed Nov 30, 2021

View reviewed changes

Dockerfile.alltools Outdated Show resolved Hide resolved

hbandura approved these changes Nov 30, 2021

View reviewed changes

Joshua Gutow added 2 commits November 30, 2021 09:56

Clean up dockerfiles

e24b62f

Clarify usage of tx comparators

0726834

This was referenced Dec 2, 2021

Review and Enable All new tests after 1.10.7 Merge #1764

Closed

Reenable golang ci linters #1765

Closed

mcortesi approved these changes Dec 2, 2021

View reviewed changes

mcortesi merged commit e0c4338 into master Dec 2, 2021

mcortesi deleted the trianglesphere/work branch December 2, 2021 01:54

erwanor mentioned this pull request Dec 8, 2021

The bind_v2 module does not use EIP155 signing #1709

Closed

eelanagaraj mentioned this pull request Apr 11, 2023

Set interpreter config to debug as well in evm.SetDebug and update tracing tests #2061

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge v1.10.7 #1695

Merge v1.10.7 #1695

trianglesphere commented Sep 16, 2021 •

edited

mcortesi left a comment

mcortesi Nov 18, 2021

mcortesi left a comment

mcortesi Nov 19, 2021

trianglesphere Nov 19, 2021

mcortesi Dec 1, 2021

mcortesi Dec 2, 2021

trianglesphere commented Nov 30, 2021

mcortesi left a comment

mcortesi Dec 1, 2021

mcortesi Dec 1, 2021

mcortesi Dec 1, 2021

mcortesi Dec 1, 2021

mcortesi Dec 1, 2021

mcortesi Dec 2, 2021

mcortesi Dec 2, 2021

mcortesi Dec 2, 2021

mcortesi Dec 2, 2021

mcortesi Dec 2, 2021

Merge v1.10.7 #1695

Merge v1.10.7 #1695

Conversation

trianglesphere commented Sep 16, 2021 • edited

Description

Notes for reviewers

Tested

Related issues

Backwards compatibility

mcortesi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcortesi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trianglesphere commented Nov 30, 2021

mcortesi left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trianglesphere commented Sep 16, 2021 •

edited