Companion for Bridges V2 - relay client part

[bkontur]

TODO

• un-dienerize before merging, uses Bridges - changes for Bridges V2 - relay client part polkadot-sdk#4494
• fix/update/align with Bridges - changes for Bridges V2 - relay client part polkadot-sdk#4494

Questions

• Do we need more testing for client V2 stuff? If so, how/what is the ultimate test?

[acatangiu]

Do we need more testing for client V2 stuff? If so, how/what is the ultimate test?

running it on Rococo<>Westend for a while should give us enough coverage

[bkontur]

relayer with client-V2 redeployed for Rococo/Westend: https://gitlab.parity.io/parity/mirrors/parity-bridges-common/-/jobs/6350508

https://grafana.teleport.parity.io/explore?schemaVersion=1&panes=%7B%22zm8%22:%7B%22datasource%22:%22P03E52D76DFE188C3%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bcontainer%3D%5C%22bridges-common-relay%5C%22%7D%22,%22queryType%22:%22range%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22P03E52D76DFE188C3%22%7D,%22editorMode%22:%22code%22%7D%5D,%22range%22:%7B%22from%22:%22now-1h%22,%22to%22:%22now%22%7D%7D%7D&orgId=1

@svyatonik

2024-05-30 13:03:20	ERROR jsonrpsee_core::client::async_client [backend]: Networking or low-level protocol error: WebSocket connection error: connection closed
2024-05-30 13:03:20	DEBUG bridge best finalized headers headers stream of BridgeHubWestend has returned `None`. It may need to be restarted
2024-05-30 13:03:20	ERROR jsonrpsee_core::client::async_client [backend]: Networking or low-level protocol error: WebSocket connection error: connection closed
2024-05-30 13:03:20	DEBUG bridge best headers headers stream of BridgeHubRococo has returned `None`. It may need to be restarted
2024-05-30 13:03:20	ERROR jsonrpsee_core::client::async_client [backend]: Networking or low-level protocol error: WebSocket connection error: connection closed
2024-05-30 13:03:20	DEBUG bridge GRANDPA justifications stream of Westend has returned `None`. It may need to be restarted
2024-05-30 13:03:20	DEBUG bridge Background task of subscription broadcaster for GRANDPA justifications stream of Westend has stopped: stream has finished
2024-05-30 13:03:20	DEBUG bridge best finalized headers headers stream of Westend has returned `None`. It may need to be restarted
2024-05-30 13:03:23	[Rococo_to_BridgeHubWestend_EquivocationDetection] ERROR bridge Could not read best finalized header number from BridgeHubWestend: ChannelError("Background task of BridgeHubWestend client has exited with result: Err(ChannelError(\"Finalized headers subscription for BridgeHubWestend has finished\"))")
2024-05-30 13:03:23	[Rococo_to_BridgeHubWestend_EquivocationDetection] INFO bridge Connecting to BridgeHubWestend node at wss://westend-bridge-hub-rpc.polkadot.io:443
2024-05-30 13:03:26	[Westend_to_BridgeHubRococo_EquivocationDetection] ERROR bridge Could not read best finalized header number from BridgeHubRococo: ChannelError("Background task of BridgeHubRococo client has exited with result: Err(ChannelError(\"Mandatory best headers subscription for BridgeHubRococo has finished\"))")
2024-05-30 13:03:26	[Westend_to_BridgeHubRococo_EquivocationDetection] INFO bridge Connecting to BridgeHubRococo node at wss://rococo-bridge-hub-rpc.polkadot.io:443
2024-05-30 13:03:58	[Westend_to_BridgeHubRococo_EquivocationDetection] TRACE bridge Read 1 finality proofs from Westend finality stream for headers in range [Some(21048090); Some(21048090)]
2024-05-30 13:03:59	[Rococo_to_BridgeHubWestend_EquivocationDetection] TRACE bridge Read 12 finality proofs from Rococo finality stream for headers in range [Some(10671149); Some(10671160)]
2024-05-30 13:04:11	[Westend_to_BridgeHubRococo_EquivocationDetection] WARN bridge Westend finality proofs stream is being started / restarted
2024-05-30 13:04:11	[Westend_to_BridgeHubRococo_EquivocationDetection] ERROR bridge Failed to subscribe to Westend justifications: ChannelError("`try_send` has failed: Closed(..)")
2024-05-30 13:04:11	[Westend_to_BridgeHubRococo_EquivocationDetection] ERROR bridge Could not connect to the Westend `FinalityProofsStream`: ChannelError("`try_send` has failed: Closed(..)")
2024-05-30 13:04:11	[Westend_to_BridgeHubRococo_EquivocationDetection] INFO bridge Connecting to Westend node at wss://westend-rpc.polkadot.io:443
2024-05-30 13:04:12	[Rococo_to_BridgeHubWestend_EquivocationDetection] TRACE bridge Read 2 finality proofs from Rococo finality stream for headers in range [Some(10671161); Some(10671162)]
2024-05-30 13:04:23	[Westend_to_BridgeHubRococo_EquivocationDetection] WARN bridge Westend finality proofs stream is being started / restarted

[svyatonik]

You're a bit out of the loop here, @bkontur :) We now use containers with names headers-a, headers-b, parachains-a, parachains-b, messages-a, messages-b. See e.g.: https://github.com/svyatonik/bridge-troubleshooting?tab=readme-ov-file#exploring-relayer-logs-online. E.g.: https://grafana.teleport.parity.io/explore?schemaVersion=1&panes=%7B%22yph%22%3A%7B%22datasource%22%3A%22P03E52D76DFE188C3%22%2C%22queries%22%3A%5B%7B%22refId%22%3A%22A%22%2C%22expr%22%3A%22%7Bpod%3D%7E%5C%22bridges-common-relay%2F.*%5C%22%2C+container%3D%5C%22headers-a%5C%22%7D%22%2C%22queryType%22%3A%22range%22%2C%22datasource%22%3A%7B%22type%22%3A%22loki%22%2C%22uid%22%3A%22P03E52D76DFE188C3%22%7D%2C%22editorMode%22%3A%22builder%22%7D%5D%2C%22range%22%3A%7B%22from%22%3A%22now-5m%22%2C%22to%22%3A%22now%22%7D%7D%7D&orgId=1

The logs you show are from the equivocation detection, which is using some old relayer version - I think we should talk to devops to also include it into our auto deployment step. Probably @serban300 could help here?

The issues from the log are caused by that:

2024-05-30 14:05:38.963	
ERROR jsonrpsee_core::client::async_client [backend]: Networking or low-level protocol error: WebSocket connection error: connection closed

Which is normal

Also (imho) it'd be better to have release when we deploy something on any chain - it'll make it easier to track relayer version/patches/... But you could go without that - that's jsut my opinion :)

[bkontur]

You're a bit out of the loop here, @bkontur :) We now use containers with names headers-a, headers-b, parachains-a, parachains-b, messages-a, messages-b.

Yes, not my best day, focusing on too many things at once :)

The logs you show are from the equivocation detection, which is using some old relayer version - I think we should talk to devops to also include it into our auto deployment step. Probably @serban300 could help here?

Will do: #3005

Also (imho) it'd be better to have release when we deploy something on any chain - it'll make it easier to track relayer version/patches/... But you could go without that - that's jsut my opinion :)

Yes, I will try to do a normal release next time. I thought that we wanted to redeploy client-V2 just from the branch and not from the master, once it will work for at least one week, we could merge to polkadot-sdk client-v2 and create release and redeploy Ro/We and P/K.

Anyway, looks like the new relayer with client V2 is working, at least I see 4 relayed message around 20:00, I didn't check logs yet, I will check next week. Also with new client-v2 I want to do: #3004