feat: remove chunking from rpc

[hansieodendaal]

Description

Removed chunking from RPC as this does not improve latency or throughput and adds code complexity.

Motivation and Context

See above.

How Has This Been Tested?

System-level tests were conducted syncing a fresh archival base node in a controlled environment over Gigabit wired LAN where only one other base node was connected. The blockchain had many full blocks and many UTXOs, as a result of a 24-hour stress test.

Header sync
No difference could be seen with header sync, as expected, due to the small message sizes.

Block sync
Blocks throughput increased during full block sync for both coins-split and normal transactions due to reduced latencies (data transfer time); up to 8% increased block throughput was measured when blocks were filled with normal transactions.

What process can a PR reviewer use to test or verify this change?

Review code changes and test data, or perform a similar comparative test.

Breaking Changes

• None
• Requires data directory on base node to be deleted
• Requires hard fork
• Other - Please specify

[github-actions]

Test Results (CI)

    3 files    120 suites   35m 58s ⏱️
1 275 tests 1 275 ✅ 0 💤 0 ❌
3 817 runs  3 817 ✅ 0 💤 0 ❌

Results for commit cc3258d.

♻️ This comment has been updated with latest results.

[github-actions]

Test Results (Integration tests)

 2 files  + 2  11 suites  +11   46m 9s ⏱️ + 46m 9s
33 tests +33  30 ✅ +30  0 💤 ±0  3 ❌ +3 
38 runs  +38  32 ✅ +32  0 💤 ±0  6 ❌ +6 

For more details on these failures, see this check.

Results for commit cc3258d. ± Comparison against base commit 1d6e0d8.

♻️ This comment has been updated with latest results.

[sdbondi]

LGTM

I'm not sure if you intended to remove chunking from the client or not. It would be a breaking change to do so but that should be fine at this point.

[sdbondi]

I didn't notice the message.to_proto() function before. That would be better to use than this

Suggested change

	proto::rpc::RpcResponse {
	request_id,
	status: message.status.as_u32(),
	flags: message.flags.bits().into(),
	payload: message.payload.to_vec(),
	}
	message.to_proto()

[sdbondi]

remove?

[hansieodendaal]

Not sure what to do with the failing response_too_big unit test. I get ServerClosedRequest instead of RpcError::RequestFailed at the first check.

[sdbondi]

Not sure what to do with the failing response_too_big unit test. I get ServerClosedRequest instead of RpcError::RequestFailed at the first check.

The problem was that the "payload too large" message wasn't being constructed properly and, without chunking, the max frame size should be >= max response size.

I pushed fixes for these

[hansieodendaal]

The problem was that the "payload too large" message wasn't being constructed properly and, without chunking, the max frame size should be >= max response size.

I pushed fixes for these

Oops, did not see that. Working on it now.

[sdbondi]

The problem was that the "payload too large" message wasn't being constructed properly and, without chunking, the max frame size should be >= max response size.
I pushed fixes for these

Oops, did not see that. Working on it now.

No problem :) readded the fix