Proposal: Coverage/Hooks API

[Furetur]

Hello!

This is an RFC. Its goal is to discuss and approve the design of a new feature. My colleagues and I aim to implement this feature, once an agreement is reached.

Feature request

Background: Currently, there is no way to collect coverage of tests that are run on the VM. There already has been an issue on this subject and a prototype implementation (#2657). That implementation suffered from a bug in debug info which just got fixed (#3412). Now we can move further

Feature: In my view, the VM would really benefit from an Execution Hooks API that would allow to programmatically register callbacks that would be invoked for each executed instruction. Coverage collection would just be a special use case of this API.

Proposed design

I propose that we:

1. Add a field that would store a hook callback to the VM struct:

type VM struct {
    ...
    executionHook func(scriptHash util.Uint160, offset int, opcode opcode.Opcode)
}

2. Define a new constructor for the VM type, e.g. NewWithExecutionHook(callback), that would create an instance of VM with the specified callback
3. Modify the step() method so that it calls the v.executionHook for each instruction

func (v *VM) step(ctx *Context) error {
+	instruction_offset := v.Context().nextip
	op, param, err := ctx.Next()
+	if v.executionHook != nil {
+		v.executionHook(v.GetCurrentScriptHash(), instruction_offset, op)
+	}
	if err != nil {
		v.state = vmstate.Fault
		return newError(ctx.ip, op, err)
	}
	return v.execute(ctx, op, param)
}

[Furetur]

I consider Execution Hooks API / Hooks API a better name for this feature, so I edited the issue

[ixje]

Just voicing my interest in this. We (COZ) would love if the end result of this implementation allows us to get the list instructions/offsets executed so we can map them back to our debug info.

[AnnaShaleva]

The design LGTM, it's more generic than the one presented in 96b5d90, so vote up from my side to implement this proposal. Also, this execution hook may be used not only for the coverage calculations, but also for custom VM logger implementation that we've discussed with @roman-khimov.

the end result of this implementation allows us to get the list instructions/offsets executed

Can easily be reached with a custom execution hook that accumulates the set of instructions inside it (which also remains me some kind of Hooks-based logger that was recently implemented in EVM, ref. https://github.com/ethereum/go-ethereum/blob/1ec7af261223d6dad9370ee8263f86347b190bab/eth/tracers/logger/logger.go#L135). So this request can easily be implemented after the described proposal implementation.