New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
traceChain individual blocks don't appear to be traced concurrently #27553
Comments
TraceChain is a producer/consumer model:
And TraceChain is IO-heavy tasks, from the logs that you provided:
The producer handles the blocks in 4s/block, that's too slow. So in my opinion, it's the bottleneck of the producer, and maybe the root cause is the IO usage. Could you please identify the usage of IOutil(eg: |
TraceChain had a change to its concurrency model recently: #24283. Basically in you're example you're using a native tracer, and in that case we process things serially. To add some context to @jsvisa's answer:
Are you running an archive node? |
Yes we are running an archive node. It is fully synced. |
Its no problem for us to move to faster storage to help the IO but the issue I am having specifically is that although the release notes of 1.8 make is seem like the blocks should processed concurrently and the logs appear to show that, they are being sent consecutively. If we moved to fast NVME it seems that we are still going to get the blocks consecutively not when they are done being traced. It seems that based on the reply of @s1na this is expected and the release notes are either incorrect or unclear for the |
Please excuse my last comment. I got confused. What we changed was the behavior of
I'm wondering if that's because of |
Thinking about this more, the producer could be blocked on the channel send. I.e. 4s is the rate of tracing a block after which a thread becomes available to process the next. It would help to see the logs from right when traceChain operation was started. |
Not sure how much in the way of logs you think would be helpful but here is what it looks like when it starts.
|
System information
Geth version:
Geth/v1.11.5-stable/linux-amd64/go1.19.1
OS & Version: Linux
Expected behaviour
Using Websockets to send the request below:
{"jsonrpc": "2.0", "id": 1, "method": "debug_subscribe", "params": ["traceChain", "0x1090f9e", "0x1091386", {"tracer": "callTracer", "tracerConfig":{"withLog": true, "onlyTopCall": true}}]}
Based on release notes for geth 1.8
individual blocks are traced concurrently, so total tracing time gets proportionally lower the more CPU cores you throw at it
I would expect to get the blocks back as they are finished (not numerical order) but instead I am getting them back as if they have been processed consecutively (in numerical order) and not concurrently.
Checked the
htop
on the machine and its not obvious that multiple cores are being used.Actual behaviour
The logs appear to show the blocks being processes concurrently but they are returned consecutively
Steps to reproduce the behaviour
The text was updated successfully, but these errors were encountered: