Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/evm: t8n support native tracers #28557

Merged
merged 19 commits into from
Dec 18, 2023
Merged

Conversation

jsvisa
Copy link
Contributor

@jsvisa jsvisa commented Nov 20, 2023

t8n is a useful tool to test state transition. Currently, we use t8n to analyze some exploit transactions, such as SaddleFinance Exploiter, a simple workflow as below:

  1. save tx's prestate into alloc.json;
  2. save tx's block metadata into env.json;
  3. save tx's rawtrasaction into txs.json;
  4. now apply the tx: ./evm t8n --trace ...;
  5. from the outputted alloc and traces to find some useful patterns.

But currently t8n only supports JSONLogger traces, we want to extend it to support all traces under eth/tracers

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
@jsvisa jsvisa requested a review from s1na as a code owner November 20, 2023 10:11
@holiman
Copy link
Contributor

holiman commented Nov 21, 2023

Interesting, I'll look into this.
Related: are you using the prestateTracer to dump the prestate, or something else?

@jsvisa
Copy link
Contributor Author

jsvisa commented Nov 22, 2023

Interesting, I'll look into this.

Related: are you using the prestateTracer to dump the prestate, or something else?

sorry for the late reply. Yes, I first use prestateTracer to dump the states(via debug_traceTransaction RPC)

Copy link
Member

@rjl493456442 rjl493456442 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nitpicks, otherwise lgtm.

Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
Signed-off-by: jsvisa <delweng@gmail.com>
@rjl493456442 rjl493456442 added this to the 1.13.6 milestone Dec 15, 2023
@jsvisa jsvisa requested a review from holiman December 15, 2023 06:57
@holiman
Copy link
Contributor

holiman commented Dec 15, 2023

Unfortunately, we cannot "unify" jsonl vs json. The --trace option configures the json-logger which emits jsonl.

jsonl - is json-lines, consisting of multiple lines of output, where each separate line is a standalone json object.
json - is json, consisting of one single json object, over arbitrary many lines.

So, when using --trace option, we need to still output jsonl files. And all other tracers emit one result at the end of execution, into json format. These two formats are incompatible. We could rename the output to, e.g. .output, but I don't know if that's an improvement? IMO let's keep them separate.

@jsvisa
Copy link
Contributor Author

jsvisa commented Dec 15, 2023

@holiman Thanks, you are right. jsonl is json lines, while json is not.

prevFile.Close()
}
traceFile, err := os.Create(path.Join(baseDir, fmt.Sprintf("trace-%d-%v.jsonl", txIndex, txHash.String())))
tracer, err := tracers.DefaultDirectory.New(ctx.String(TraceTracerFlag.Name), nil, config)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When you changed this to instantiate the tracer earlier, you made it so that every transaction uses the same tracer instance. I don't think that's how the lifecycle of a tracer works - all other tracer-usage instantiates a new tracer for each transaction. (correct me if I'm wrong, @s1na )

Please revert this change so the tracer is instantiated in getTracer

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, let's instantiate inside the getTracer, one tracer for each tx

Signed-off-by: jsvisa <delweng@gmail.com>
This reverts commit 931c814.

Signed-off-by: jsvisa <delweng@gmail.com>
@@ -28,12 +28,15 @@ import (
var (
TraceFlag = &cli.BoolFlag{
Name: "trace",
Usage: "Output full trace logs to files <txhash>.jsonl",
Usage: "Output full trace logs to a file named: trace-<txIndex>-<txHash>.json or trace-<txIndex>-<txHash>.jsonl",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Usage: "Output full trace logs to a file named: trace-<txIndex>-<txHash>.json or trace-<txIndex>-<txHash>.jsonl",
Usage: "Output json opcode traces to files (trace-<txIndex>-<txHash>.jsonl),

IMO the --trace configures the json tracer. For the other ones, it's implied. Why would you want to run a calltracer if it does not produce any output?

I'll push a fix

@holiman holiman merged commit c18c5c3 into ethereum:master Dec 18, 2023
2 of 3 checks passed
@jsvisa jsvisa deleted the evm-t8n-calltracer branch December 20, 2023 03:44
Doozers pushed a commit to kilnfi/pgeth that referenced this pull request Dec 22, 2023
This change implements ability for the `evm t8n` tool to use custom tracers; either 'native' golang tracers or javascript tracers.
Dergarcon pushed a commit to specialmechanisms/mev-geth-0x2mev that referenced this pull request Jan 31, 2024
This change implements ability for the `evm t8n` tool to use custom tracers; either 'native' golang tracers or javascript tracers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants