Gas Consumption Framework

[robert-zaremba]

Summary

It would be useful to have a reference fee values and a mechanism to assign fees for new operations (gas consumption, don't confuse with gas prices).

In SDK, the gas consumption values (amount of gas per certain operations) are spread across all packages. It's hard to have any grasp about current fee values.
Without a reference fee framework, we can't correctly asses gas consumption values for new operations. Furthermore it's almost impossible to easily scale these values by some factor.

Context

While productionizing and analyzing x/fee_grant and x/authz we had a problem of creating and assessing new fee values.

Goals

1. Create a reference table with basic operations and their costs. Some of this is already done: KVGasConfig, DefaultSigVerifyCostSecp256k1. However, there is no single baseline and it's impossible to imports some of this values without creating import cycles or config management difficulties.
2. Wrap the table into a gas free framework. It should be an extensible interface which allows to load and scale fees.
3. Make it easy to reuse or specialize the framework by SDK chains: any SDK chain should apply it's own fee baseline and extend it, without modifying any core package.
4. Measure fee usage: we can integrate a monitoring tool with the gas free framework and measure which types of fees are mostly use.

[aaronc]

Thanks @robert-zaremba. It would be great if we had the fee table configurable via params and available to all modules. Then, x/authz or x/feegrant could depend on it and any param tweaks would be against the base gas fee table rather than for each module. This model I think would work well for generic use cases like iterating through loops rather than needing to do everything case by case.

[robert-zaremba]

The EVM was designed to compute the operation cost while processing all operations. Go is not designed for such thing and hacking it, or paring every line of code with ConsumeGas would be an overkill.

We should focus on higher-level functions, which will be used by keepers.

[amaury1093]

We should focus on higher-level functions, which will be used by keepers.

Does that mean the gas table will have 1 entry per keeper function? Or only a subset of keeper functions will have gas consumption (to not overload the table)?

IMO, with ADR-033 keepers are meant to be internal to a module, so exposing their gas consumption in a public table and in params seems weird. With this thinking, what makes the more sense to me is 1 gas table entry == 1 Msg's gas consumption. There could also be a couple more entries for antehandlers gas consumption.

[robert-zaremba]

IMHO, it shouldn't contain keeper functions, but rather a reference operations, eg: route, sort, sigEd25519Verify, BLS382Verify, SimpleLoopStep, ...

Not sure if we want to register Msgs costs there. What do you think?

[amaury1093]

Got it, I understood the level of granulariy you're aiming.

[blewater]

IMHO, it shouldn't contain keeper functions, but rather a reference operations, eg: route, sort, sigEd25519Verify, BLS382Verify, SimpleLoopStep, ...

Not sure if we want to register Msgs costs there. What do you think?

Predictable Msg costs are something we are presently implementing for our Dex orders. If the scope of this is high level enough to include Msgs it could be very interesting for what it implies for user engagement. This is an early-stage proposal from what I understand, right?

[tac0turtle]

This sounds good to me. Do you imagine modules registering this as a param or just as compile time?

Does this include both these items?

1. Allowing apps to register fee models (EIP-1559, fee base line)
2. Allowing apps to register specific consumption for a msg

[robert-zaremba]

Thinking more... my motivation here is to have a reference gas consumptions for SDK core operations. However it might be interesting to register more reference operation by modules which are beyond our (SDK team) control, what do you think?

[tac0turtle]

Okay, so this is mainly adding functions for people to use. Seems like a good idea, but makes me quesetion why do a bunch of functions that aren't used in the sdk need to live in the sdk. We can provide what the sdk uses and tell users to develop their own if they want. The API surface of the sdk is huge already, we should be careful to not keep increasing it just because.

[robert-zaremba]

By registering reference operation I meant to register a kind / name, rather than registering a specific function / exec. But I don't know if that is needed. Does this solves you concern, @marbar3778 ?

[fedekunze]

I think registering gas per core operations makes sense. One of the main problems rn is that the gas consumption is not configurable and all its values are hardcoded (idk why is it called config if you can't even update it...):

// KVGasConfig returns a default gas config for KVStores.
func KVGasConfig() GasConfig {
	return GasConfig{
		HasCost:          1000,
		DeleteCost:       1000,
		ReadCostFlat:     1000,
		ReadCostPerByte:  3,
		WriteCostFlat:    2000,
		WriteCostPerByte: 30,
		IterNextCostFlat: 30,
	}
}

[amaury1093]

One argument against registering gas per core operation is that sometimes a Msg means future execution. For example: MsgVote adds vote to state, but the actual heavier computation (tallying, i.e. counting all votes) happens on EndBlock.

With a gas per core operation, I don't see how we could account for EndBlock gas consumption when sending a MsgVote. On the other hand, with a gas per Msg, we could arbitrarily set a higher gas to account for future EndBlock computation.

[robert-zaremba]

For messages, I think it's hard to make a Msg level fixed fee - I think it's easier to think about operations when we process message.
cc: @AmauryM , @marbar3778

[iramiller]

Another angle on gas consumption is the time required to process each type of msg.

As "time required to process" would be highly variable based on the hardware the node is running on and thus non-deterministic this approach would be tough to get right... but presuming there is a way to do it we could potentially determine a continuous query of "median time required to process" for each type of msg based on the last 100 blocks that contained the message.

The timing information from this query could then be fed back into the system to maintain a fee table.

[amaury1093]

For comparison, Substrate uses execution time of each Msg on a reference hardware to determine gas consumption (called weights): https://substrate.dev/docs/en/knowledgebase/learn-substrate/weight. Weights can also be function of Msg inputs (e.g. here).

The weight-to-fee function (be it a simple multiplier, a polynomial...) is stored and governed on chain.

[robert-zaremba]

The most important is the ratio between different operations. This shouldn't change between different hardware stations, unless you are using something old - in this case you will pay a small time penalty for your hardware (which you will anyway, because all operations will take more time, but some will be proportionally more longer).