eth_call rate very poor

[michelescippa]

System information

Geth version: v1.13.14-stable-2bd6bd01
CL client & version: lighthouse@v5.1.2-modern
OS & Version: Linux

After installing on i3en.xlarge AWS instance, configured with the NVME (XFS, noatime), performances on eth_call are very very poor.

I made lots of requests to fetch same data and seems that the node starts to respond quickly then on the subsequent seems to wait before execution.

First calls are executed in less than 20-30 ms, this time fastly increase to 1s, 2s... to the timeout.

The feeling is that it's not able to handle concurrent requests, but waits for the formers to complete

I made the same requests to a QuickNode enpdoint, which has installed the same geth version and the node responds very quickly as expected, all calls returned in less than 150ms.

Tried to tune all parameters, cache etc.
This is my current configuration:

--config "/root/.ethereum/config.toml" \ --mainnet \ --log.rotate \ --syncmode snap \ --gcmode full \ --authrpc.jwtsecret="/root/.ethereum/jwtsecret" \ --authrpc.addr "0.0.0.0" \ --authrpc.vhosts "*" \ --http \ --http.addr "0.0.0.0" \ --http.corsdomain "*" \ --ws \ --ws.addr "0.0.0.0" \ --ws.origins "*" \ --cache 16384 \ --cache.preimages \ --maxpeers 5 \ --metrics \ --metrics.addr "0.0.0.0" \ --metrics.port 6060 \ --metrics.expensive \ --rpc.allow-unprotected-txs \ --state.scheme=path \

Tried to disabled snapshot mode, but the performances are much worse also in block execution time (5-6secs vs 120-140ms).

[michelescippa]

Strange behavior, tried to set execution timeout to 1m, the limit is set as you can see from the log but the execution is aborted after 10 secs.

WARN [03-22|19:10:37.741] Served eth_call conn=172.31.22.121:40774 reqid=1566 duration=9.056983976s err="execution aborted (timeout = 1m0s)" WARN [03-22|19:10:37.817] Served eth_call conn=172.31.22.121:39908 reqid=1617 duration=10.031993551s err="execution aborted (timeout = 1m0s)" WARN [03-22|19:10:37.866] Served eth_call conn=172.31.22.121:40984 reqid=1607 duration=9.094709907s err="execution aborted (timeout = 1m0s)"

[learnerLj]

There is a parameter to contral the max concurrency for RPC. Additionnally, eth_call is considerably expensive.

[michelescippa]

There is a parameter to contral the max concurrency for RPC. Additionnally, eth_call is considerably expensive.

Thank you for your answer.
After many tries I found that the problem is in concurrent calls.
BTW I cannot found the parameter to limit them, I saw only --rpc.batch-request-limit but I interpreted it as it is for batch requests, may I'm wrong.

Can you confirm?

[learnerLj]

BTW I cannot found the parameter to limit them, I saw only --rpc.batch-request-limit but I interpreted it as it is for batch requests, may I'm wrong.

Sorry, It is my guess. And When I look it the code, I do not find the according limit. Instead, eth/tracers/api.go provide limits on each call trace, such as the amount of time a single transaction can execute, 5 seconds. if your tx is unlimited, it may exceed these limits. I am not sure whether there are limits on EVM interpreter, but the concurrency number 1000, is ok as my machine has enough source.

const (
	// defaultTraceTimeout is the amount of time a single transaction can execute
	// by default before being forcefully aborted.
	defaultTraceTimeout = 5 * time.Second

	// defaultTraceReexec is the number of blocks the tracer is willing to go back
	// and reexecute to produce missing historical state necessary to run a specific
	// trace.
	defaultTraceReexec = uint64(128)

	// defaultTracechainMemLimit is the size of the triedb, at which traceChain
	// switches over and tries to use a disk-backed database instead of building
	// on top of memory.
	// For non-archive nodes, this limit _will_ be overblown, as disk-backed tries
	// will only be found every ~15K blocks or so.
	defaultTracechainMemLimit = common.StorageSize(500 * 1024 * 1024)

	// maximumPendingTraceStates is the maximum number of states allowed waiting
	// for tracing. The creation of trace state will be paused if the unused
	// trace states exceed this limit.
	maximumPendingTraceStates = 128
)

// TraceCall lets you trace a given eth_call. It collects the structured logs
// created during the execution of EVM if the given transaction was added on
// top of the provided block and returns them as a JSON object.
// If no transaction index is specified, the trace will be conducted on the state
// after executing the specified block. However, if a transaction index is provided,
// the trace will be conducted on the state after executing the specified transaction
// within the specified block.
func (api *API) TraceCall(ctx context.Context, args ethapi.TransactionArgs, blockNrOrHash rpc.BlockNumberOrHash, config *TraceCallConfig) (interface{}, error) {
	.....
	return api.traceTx(ctx, msg, new(Context), vmctx, statedb, traceConfig)
}

[joevarghese]

anyone find a solution for this?