Heavy concurrent use of web3 causes crashes

[marmistrz]

For the example, I'm using the public API key from Alchemy for testing rate limits, see https://docs.alchemy.com/alchemy/documentation/throughput

Example code:

async fn batch_worker<T: BatchTransport>(web3: Web3<Batch<T>>) {
    let fut = web3.eth().block_number();
    web3.transport()
        .submit_batch()
        .await
        .expect("submitting batch");
    fut.await.expect("block_number");
}

#[tokio::main]
async fn main() {
    env_logger::init();
    let node = "wss://eth-mainnet.alchemyapi.io/v2/J038e3gaccJC6Ue0BrvmpjzxsdfGly9n";
    let ws = web3::transports::WebSocket::new(node)
        .await
        .expect("building ws");
    let batch = Batch::new(ws);
    let web3 = Web3::new(batch);

    let num_workers = 100;
    let handles: Vec<_> = (0..num_workers)
        .map(|_| tokio::spawn(batch_worker(web3.clone())))
        .collect();
    futures::future::join_all(handles).await;
}

Running this with cargo run will cause abnormal request grouping:

[2022-05-30T12:58:56Z DEBUG web3::transports::ws] [0] Calling: [], "[]"
[2022-05-30T12:58:56Z DEBUG web3::transports::ws] [17] Calling: [{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":17},{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":18}], "[{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":17},{\"jsonrpc\":\"2.0\",\"method\":\"eth_blockNumber\",\"params\":[],\"id\":18}]"

and further panics such as:

thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', /home/marcin/proj/rust-web3/src/transports/ws.rs:332:43

or

thread 'tokio-runtime-worker' panicked at 'submitting batch: Transport(Message("Cannot send request. Internal task finished."))', src/main.rs:168:10

This only happens for batch transports

[marmistrz-stc]

This was a combination of two things:

• an issue introduced in an umerged PR Report response parse errors to avoid freezes #639
• the fact that cloned instances of Batch<T> share the pending/unsent requests queues. This should be documented. I'll open a documentation PR in a while

[palinko91]

• the fact that cloned instances of Batch<T> share the pending/unsent requests queues. This should be documented. I'll open a documentation PR in a while

I never experienced parsing error, but With a simple code like this:

let mut subscription = web3s.eth_subscribe().subscribe_logs(filter.clone()).await?;

let test =  subscription.next().await;
println!("{:?}", test);

I got None right now, instead of waiting for the filter requirtments to met. And if I try to loop and using web3 then I get this code easily also.

Transport("Cannot send request. Internal task finished.")

[marmistrz]

Please open a separate issue for this.