failed to fetch internal transactions for blocks: :closed

[islishude]

Describe your issue here.

Environment

• Deployment type (Manual/Docker/Docker-compose): docker-compose
• Elixir & Erlang/OTP versions (elixir -version): latest
• Node JS version (node -v): latest
• Operating System: Ubuntu 20.04
• Blockscout Version/branch/commit: master
• Archive node type && version (Erigon/Geth/Nethermind/Ganache/?): Geth

Steps to reproduce

ENV

      ETHEREUM_JSONRPC_DEBUG_TRACE_TRANSACTION_TIMEOUT: '10m'
      INDEXER_DISABLE_PENDING_TRANSACTIONS_FETCHER: 'true'
      INDEXER_DISABLE_BLOCK_REWARD_FETCHER: 'true'
      INDEXER_DISABLE_EMPTY_BLOCK_SANITIZER: 'true'
      ETHEREUM_JSONRPC_DISABLE_ARCHIVE_BALANCES: 'true'
      INDEXER_INTERNAL_TRANSACTIONS_CONCURRENCY: '1'

Log

2022-12-16T03:40:36.066 application=indexer fetcher=block_catchup first_block_number=4189779 last_block_number=430100 missing_block_count=0 shrunk=false [info] Index already caught up.
2022-12-16T03:40:36.066 application=indexer fetcher=block_catchup [info] Checking if index needs to catch up in 5000ms.
2022-12-16T03:40:36.174 application=indexer fetcher=internal_transaction count=10 error_count=10 [error] failed to fetch internal transactions for blocks: :closed

Expected behaviour

Tell us what should happen.

Actual behaviour

Tell us what happens instead.

[vbaranov]

@islishude you can also try decreasing INDEXER_INTERNAL_TRANSACTIONS_BATCH_SIZE. By default, it is 10.

[islishude]

Ok, let me try out with it

[islishude]

Not works. I have changed it to 1

[islishude]

I don't know what the closed error means, It's weird

[vbaranov]

Is debug module enabled on Geth node? Can you request Geth node with debug_traceTransaction method directly for any transaction?

[islishude]

yes, it's enabled

[vbaranov]

Please post a response from debug_traceTransaction for any transaction.

An example (replace tx hash and node endpoint):

curl -sL \
  -H "Content-Type:application/json;charset=utf-8" \
  -d "{
      \"jsonrpc\": \"2.0\",
      \"id\": 1,
      \"method\": \"debug_traceTransaction\",
      \"params\": [
          \"0x...\",
          { \"tracer\": \"callTracer\"}
      ]
  }" \
http://...

[islishude]

I'm sure it's enabled.

some txes have raw trace, some txes don't

[vbaranov]

Then probably, the node lack resources under the loading. Try to increase resources for it.

[islishude]

I run the blockscout on an aws c5.4xlarge instance, load is not high now

1.87, 1.86, 1.86

[islishude]

blockscout indexer recv_timeout is 30 mins

    transport_options: [
      http: EthereumJSONRPC.HTTP.HTTPoison,
      url: System.get_env("ETHEREUM_JSONRPC_HTTP_URL"),
      http_options: [recv_timeout: :timer.minutes(30), timeout: :timer.minutes(1), hackney: hackney_opts]
    ],

tracer timeout is also 30 mins

ETHEREUM_JSONRPC_DEBUG_TRACE_TRANSACTION_TIMEOUT: '30m'

geth's rpc server writeTimeout is 30 mins

I still get this error, and I don't get any timeout error from geth

2023-01-04T00:36:22.728 application=indexer fetcher=internal_transaction count=1 error_count=1 [error] failed to fetch internal transactions for blocks: :closed

I don't think it's related to timeout setting.

maybe it's a bug from httpoison?

edgurgel/httpoison#326

[hundredark]

blockscout indexer recv_timeout is 30 mins

    transport_options: [
      http: EthereumJSONRPC.HTTP.HTTPoison,
      url: System.get_env("ETHEREUM_JSONRPC_HTTP_URL"),
      http_options: [recv_timeout: :timer.minutes(30), timeout: :timer.minutes(1), hackney: hackney_opts]
    ],

tracer timeout is also 30 mins

ETHEREUM_JSONRPC_DEBUG_TRACE_TRANSACTION_TIMEOUT: '30m'

geth's rpc server writeTimeout is 30 mins

I still get this error, and I don't get any timeout error from geth

2023-01-04T00:36:22.728 application=indexer fetcher=internal_transaction count=1 error_count=1 [error] failed to fetch internal transactions for blocks: :closed

I don't think it's related to timeout setting.

maybe it's a bug from httpoison?

edgurgel/httpoison#326

Hi, have you solve this problem?

[phpmac]

Is it solved?