eth, miner: remove duplicated code #23256

rjl493456442 · 2021-07-23T06:25:17Z

This PR removes the duplicated code in the catalyst package, reuse the miner package code instead.

Highlight the critical changes in this PR:

Offer one more API in the miner package called GetSealingBlock which will assemble and return the sealing block based on the given parameters.
Sealing environment is abstracted out, all the functions will receive the sealing environment for further operations
Uncle blocks management is changed for sake of simplification

holiman · 2021-07-27T12:44:04Z

This touches some core logic in the miner, I think we should have a review call to discuss this one.

gballet

I agree with Martin, it seems correct (and I like that you broke some functions down into some with an explicit title) even though we should discuss this during a call. I have a couple nitpicks until then.

miner/worker_test.go

miner/worker.go

rjl493456442 · 2021-07-29T03:04:53Z

Yes @holiman and @gballet . I won't want to merge this PR now, definitely after the fork. And we can schedule a review call for it, especially for some tiny details in the miner refactoring.

miner/worker.go

badbitses · 2021-08-22T14:32:16Z

I want to help

badbitses · 2021-08-22T14:35:21Z

Who left and why

miner/worker.go

jwasinger · 2021-08-22T23:46:15Z

Who left and why

Hey @badbitses . This PR is mostly complete. If you are interested in contributing, I would advise to take a look through the open issues and see what piques your interest.

miner/worker.go

holiman · 2021-11-29T10:13:30Z

miner/worker.go

+func (w *worker) getSealingBlock(parent common.Hash, timestamp uint64, coinbase common.Address) (*types.Block, error) {
+	req := &getWorkReq{
+		params: &generateParams{
+			timestamp:  timestamp,
+			forceTime:  true,
+			parentHash: parent,
+			coinbase:   coinbase,
+			noUncle:    true,
+			noExtra:    true,
+		},
+		result: make(chan *types.Block, 1),
+	}


It seems that this method is only used for PoS blocks? Which makes sense, because it has noUncle and noExtra set to true, so it's not really abstract.
I find it a bit confusing to have this seemingly abstract block sealer mode for catalyst, whereas the ethash and clique engines use some other path to produce a block. Is the intent to later on align these three engines?

The main reason for catalyst has different path to produce block is it's rpc-event-based engine. While for ethash and clique they are chain-event-based engine, which just create blocks non-stop.

It's still ugly for them to have different paths, but we can find some proper solution to merge these paths I guess.

miner/worker.go

eth/catalyst/api.go

holiman

Looks ok, but I'm a bit confused by all the changes to the catalyst part.
Also, what I"m a bit scared about is coinbase - it would be bad if these changes make setting the coinbase not "stick".
So if a miner is mining, they need to be able to set the coinbase via,

startup param,
rpc,
- before starting mining,
- and during mining
- and when mining is requested, but during sync, so mining is disabled

Do we have tests for these? Have you experimented anything with these scenarios?

karalabe · 2022-01-05T08:44:37Z

miner/worker.go

+		//
+		// The maximum acceptable reorg depth can be limited by the finalised block
+		// somehow. TODO(rjl493456442) fix the hard-coded number here later.
+		state, err = w.eth.StateAtBlock(parent, 1024, nil, false, false)


Is this really so in practice? We're going to hold onto the latest 128 states either way for snap sync and reorgs. This clause would only trigger if someone requests mining on something super old. Is that a viable requirement?

Not sure, but I guess so. Consensus layer might reorg to some old branch and require us to build a block there. If the chain is not finalized for very long time, so the deep reorg is expected?

I think so too, since there is not really a guarantee that we won't reorg very deeply with non-finalization

Sure, but we'd get a sethead first from the beacon chain before mining on top, wouldn't we? Which would do the reorg and reconstruction of the needed state? Maybe I'm wrong tho'

MariusVanDerWijden

Some small nits, looks pretty solid to me

eth/catalyst/api.go

miner/stress/beacon/main.go

MariusVanDerWijden · 2022-01-17T09:27:16Z

miner/worker.go

+		//
+		// The maximum acceptable reorg depth can be limited by the finalised block
+		// somehow. TODO(rjl493456442) fix the hard-coded number here later.
+		state, err = w.eth.StateAtBlock(parent, 1024, nil, false, false)


I think so too, since there is not really a guarantee that we won't reorg very deeply with non-finalization

miner/worker.go

MariusVanDerWijden · 2022-01-17T12:21:45Z

I've asked Pari to run this code on a validator on kintsugi. Would be great to also put it on a clique miner maybe? cc @holiman

holiman · 2022-01-17T12:28:58Z

Sure

holiman · 2022-01-17T13:07:30Z

On rinkeby-aws-eu-west-3-001: Geth/v1.10.16-unstable-cc7f6269-20220112/linux-amd64/go1.17.6

MariusVanDerWijden · 2022-01-21T09:57:13Z

eth/catalyst/api.go:80: File is not goimports-ed (goimports)
preparedBlocks *payloadQueue // preparedBlocks caches payloads (*ExecutableDataV1) by payload ID (PayloadID)

karalabe

LGTM!

* eth, miner: remove duplicated code * eth/catalyst: remove unneeded code * miner: keep update pending state even the Merge is happened * eth, miner: rebase * miner: fix tests * eth, miner: address comments from marius * miner: use empty zero randomness for pending blocks after the merge * eth/catalyst: gofmt * miner: add warning log for state recovery * miner: ignore uncles for post-merge blocks Co-authored-by: Péter Szilágyi <peterke@gmail.com>

rjl493456442 requested a review from gballet as a code owner July 23, 2021 06:25

rjl493456442 changed the title ~~[WIP] eth, miner: remove duplicated code~~ eth, miner: remove duplicated code Jul 26, 2021

gballet reviewed Jul 28, 2021

View reviewed changes

miner/worker_test.go Outdated Show resolved Hide resolved

miner/worker_test.go Outdated Show resolved Hide resolved

miner/worker.go Outdated Show resolved Hide resolved

rjl493456442 force-pushed the abstract-miner branch from 282f3c8 to 78212c8 Compare August 9, 2021 03:27

jwasinger mentioned this pull request Aug 11, 2021

miner: support multiple custom block collation strategies evaluated concurrently #23383

Closed

jwasinger reviewed Aug 19, 2021

View reviewed changes

miner/worker.go Outdated Show resolved Hide resolved

jwasinger mentioned this pull request Aug 19, 2021

miner: support custom block collation strategies #23421

Closed

jwasinger reviewed Aug 22, 2021

View reviewed changes

miner/worker.go Outdated Show resolved Hide resolved

jwasinger reviewed Aug 24, 2021

View reviewed changes

miner/worker.go Outdated Show resolved Hide resolved

rjl493456442 force-pushed the abstract-miner branch from 78212c8 to 6239e0f Compare August 25, 2021 06:21

jwasinger mentioned this pull request Sep 16, 2021

Collator v3 jwasinger/go-ethereum#1

Closed

rjl493456442 force-pushed the abstract-miner branch from 8e4e457 to 65ca904 Compare October 11, 2021 05:05

rjl493456442 force-pushed the abstract-miner branch from 65ca904 to 20936a1 Compare November 29, 2021 06:55

holiman reviewed Nov 29, 2021

View reviewed changes

miner/worker.go Outdated Show resolved Hide resolved

fjl added the status:triage label Dec 7, 2021

rjl493456442 force-pushed the abstract-miner branch from 4124b97 to 7c56c0a Compare December 9, 2021 03:26

rjl493456442 requested a review from karalabe as a code owner December 9, 2021 03:26

ligi removed the status:triage label Dec 14, 2021

holiman reviewed Dec 14, 2021

View reviewed changes

eth/catalyst/api.go Show resolved Hide resolved

holiman reviewed Dec 14, 2021

View reviewed changes

rjl493456442 force-pushed the abstract-miner branch from e22dadd to 8d636fd Compare January 5, 2022 04:05

karalabe reviewed Jan 5, 2022

View reviewed changes

rjl493456442 force-pushed the abstract-miner branch from 7574518 to 84e85be Compare January 11, 2022 08:27

MariusVanDerWijden reviewed Jan 17, 2022

View reviewed changes

rjl493456442 force-pushed the abstract-miner branch from 6e950e4 to c9fd441 Compare January 21, 2022 06:46

rjl493456442 and others added 8 commits January 24, 2022 06:52

eth, miner: remove duplicated code

9a822db

eth/catalyst: remove unneeded code

69e7e1c

miner: keep update pending state even the Merge is happened

2473b06

eth, miner: rebase

60f0a6f

miner: fix tests

2914038

eth, miner: address comments from marius

3ff0e58

miner: use empty zero randomness for pending blocks after the merge

269bc0b

eth/catalyst: gofmt

5287292

karalabe force-pushed the abstract-miner branch from c9fd441 to 5287292 Compare January 24, 2022 04:55

rjl493456442 added 2 commits January 24, 2022 14:31

miner: add warning log for state recovery

a2480f1

miner: ignore uncles for post-merge blocks

175eb47

karalabe approved these changes Jan 24, 2022

View reviewed changes

karalabe added this to the 1.10.16 milestone Jan 24, 2022

karalabe merged commit 78636ee into ethereum:master Jan 24, 2022

holiman mentioned this pull request Jan 27, 2022

miner: Data race in TestGenerateBlockAndImportEthash #24299

Closed

sidhujag pushed a commit to syscoin/go-ethereum that referenced this pull request Jan 28, 2022

eth, miner: use miner for post-merge block production (ethereum#23256)

7f9629e

This was referenced Sep 23, 2022

Metadium to master METADIUM/go-metadium#24

Closed

Metadium to master METADIUM/go-metadium#25

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eth, miner: remove duplicated code #23256

eth, miner: remove duplicated code #23256

rjl493456442 commented Jul 23, 2021 •

edited

holiman commented Jul 27, 2021

gballet left a comment

rjl493456442 commented Jul 29, 2021

badbitses commented Aug 22, 2021

badbitses commented Aug 22, 2021

jwasinger commented Aug 22, 2021

holiman Nov 29, 2021

rjl493456442 Jan 5, 2022 •

edited

holiman left a comment

karalabe Jan 5, 2022

rjl493456442 Jan 5, 2022 •

edited

MariusVanDerWijden Jan 17, 2022

karalabe Jan 24, 2022

MariusVanDerWijden left a comment

MariusVanDerWijden Jan 17, 2022

MariusVanDerWijden commented Jan 17, 2022

holiman commented Jan 17, 2022

holiman commented Jan 17, 2022

MariusVanDerWijden commented Jan 21, 2022

karalabe left a comment

eth, miner: remove duplicated code #23256

eth, miner: remove duplicated code #23256

Conversation

rjl493456442 commented Jul 23, 2021 • edited

holiman commented Jul 27, 2021

gballet left a comment

Choose a reason for hiding this comment

rjl493456442 commented Jul 29, 2021

badbitses commented Aug 22, 2021

badbitses commented Aug 22, 2021

jwasinger commented Aug 22, 2021

holiman Nov 29, 2021

Choose a reason for hiding this comment

rjl493456442 Jan 5, 2022 • edited

Choose a reason for hiding this comment

holiman left a comment

Choose a reason for hiding this comment

karalabe Jan 5, 2022

Choose a reason for hiding this comment

rjl493456442 Jan 5, 2022 • edited

Choose a reason for hiding this comment

MariusVanDerWijden Jan 17, 2022

Choose a reason for hiding this comment

karalabe Jan 24, 2022

Choose a reason for hiding this comment

MariusVanDerWijden left a comment

Choose a reason for hiding this comment

MariusVanDerWijden Jan 17, 2022

Choose a reason for hiding this comment

MariusVanDerWijden commented Jan 17, 2022

holiman commented Jan 17, 2022

holiman commented Jan 17, 2022

MariusVanDerWijden commented Jan 21, 2022

karalabe left a comment

Choose a reason for hiding this comment

rjl493456442 commented Jul 23, 2021 •

edited

rjl493456442 Jan 5, 2022 •

edited

rjl493456442 Jan 5, 2022 •

edited