Simulator ethereum/engine: Missing tests from Mikhail's Test Coverage

[marioevz]

On April 4th @mkalinin shared a list of test coverage points for the Merge, Engine API.

Opening this issue to cross-reference and fill out any gaps between the currently implemented tests in the ethereum/engine simulator and the test coverage list.

Link: The Merge. Test coverage

Engine API Gaps:

• Improve INVALID_BLOCK_HASH coverage by expanding Bad Hash on NewPayload to test:
  • EL Not SYNCING: Invalid hash payload extending side chain
  • EL SYNCING: Invalid hash payload extending canonical chain
  • EL SYNCING: Invalid hash payload extending side chain chain
• Improve all Invalid * NewPayload test cases by querying the Eth JSON-RPC to make sure that the invalid payload is not available.
• Improve all Invalid * NewPayload test cases by adding the P0 <- INV_P <- P1, P1 == INVALID check (TBD).
• Improve all Invalid * NewPayload test cases by adding a fcU to a valid payload built on top of the invalid payload.
• Improve GetPayload with a random PayloadID and verify the error code.
• Add testcase: Send fcU with an unknown headBlockHash to set to Syncing mode. Send NewPayload with parentHash equal to the headBlockHash of the fcU. Response must be SYNCING since the payload does extend the canonical chain but the previous payload is necessary to validate.
• Improve Invalid * NewPayload test case to also test an invalid payload as transition block.
• Add testcase: EL was SYNCING and during the sync it’s been fed with INV_TB <- P1, where INV_TB is invalid terminal block and P1 is a transition payload built on top of this block. Then EL receives newPayload(P2), where P2 is a child of P1 and MUST respond with INVALID_TERMINAL_BLOCK
• Add testcase: EL was SYNCING and during the sync it’s been fed with TB <- INV_P1, where INV_P1 is INVALID payload of a transition block. Then EL receives forkchoiceUpdated(INV_P1) – transition MUST NOT happen
• Add testcase: Similar to unknownSafeBlockHash/unknownFinalizedBlockHash, add zeroesSafeBlockHash/zeroesFinalizedBlockHash.
• Add testcase: EL was SYNCING and during the sync it’s been fed with INV_TB <- P1, where INV_TB is invalid terminal block and P1 is a transition payload built on top of this block. Then EL receives forkchoiceUpdated(P1) and MUST respond with INVALID_TERMINAL_BLOCK
• Improve CL Mocker by adding a check that the new headers generated by every payload contain the correct uncleHash, difficulty, mixDigest, nonce.

Transition Gaps:

• Check on INVALID_TERMINAL_BLOCK tests that the PoW progresses.
• On Latest Block after NewPayload/HeadBlockHash/SafeBlockHash/FinalizedBlockHash, verify also for transition block.
• Verify "safe" and "finalized" JSON-RPC Block endpoints on forkchoiceUpdated calls with SafeBlockHash/FinalizedBlockHash.
• Add testcase: Send newPayload(P) where TB <- P, skip fcU(P), then send newPayload(P1), where P <- P1, and lastly fcu(P1).
• Add testcase: First client builds on top of a different Terminal Block TB1, CL builds a couple of blocks, but then the chain re-orgs to a different Terminal Block TB2 from a second client, and the CL continues building on top of this later block. First client must re-org.