MultiMsg Txs Piping Outputs

[sunnya97]

In the SDK, we support "multimsg txs" which allow multiple sdk.Msgs to be packed into a single atomic tx. This is very useful for many ux flows. However, the usability of this is severely limited because the inputs of all msgs needs to be known at tx creation, meaning that subsequent msgs cannot be based on the input of previous msgs. To give a concrete example, imagine a user on Osmosis wanted to do a swap of OSMO to ATOM and then IBC the Atoms to the Cosmos Hub.

They would ideally want to construct a multimsg tx that looks something like:

sdk.Msg[{Swap 5 OSMO to _X_ ATOM, IBC _X_ ATOM to Cosmos Hub}]

However, X cannot be known at the time of creating the tx, because of slippage and how AMMs work.

So ideally, what we would want is a way of piping an output from one msg into the input of a later msg.

To do this, we could make use of the MsgResponse of the msgs. For example, in Osmosis's MsgSwapResponse, we include the tokenOutAmount. If there was a way that in the IBC Transfer msg, we could signal to use the MsgSwapResponse.tokenOutAmount at runtime instead of a pre-filled number.

In the specific above case, we could hack this in because both the output and input are strings, and so we could special case a string format to signal doing pipelining during runTx (i.e. something like "|MsgSwapResponse.tokenOutAmount").

However, this is not fully generalizable if the types were not strings, and were instead uint64 (for example an id of some sort).

How would we make this possible? Would it require turning every field in all proto structs for Msgs into some type of new interface or type? Would love to hear if anyone has any other ideas?

[aaronc]

Yeah, this would be cool. I designed something to do this a while ago but forgot to post it. My idea was to combine a small expression language like CEL with state charts. I'll try to dig up what I started writing and post it sometime this week

[hxrts]

do you mind posting this aaron? I remember discussing this with you, would like to read any notes you have. If I remember correctly, the CEL strategy was all computable in advance, so gas usage could also be deterministic.

[aaronc]

Let me try to briefly summarize:

• we would start with a small not-turing complete expression language with protobuf as the type system, for which CEL is a good candidate,
• and some state chart specification, using SCXML as the starting point,
• state chart actions (send/invoke) would be executing Msg`s (or maybe even queries),
• state chart events would be Msg completion with the result data (in the future there could even be async triggering - i.e. completion of Msg's over IBC or listenining for state machine events)
• a small amount of local data could be assigned in state transitions

In my mind, the advantages of this sort of strategy over say just allowing the execution of scripts (solidity, javascript, rust, etc.) are:

• CEL actually works natively with protobuf types
• we separate the expression language as something non-turing complete (CEL) from the recursive/event-based part of the system (state charts)
• this allows us to visualize the recursive/event-based part as a graph which I think is easier for humans to reason about
• intentionally limits and constrains expressivity to make it easier to introspect
• is suitable for transient one-off scripts (like in the example above or other things we discussed in the gov/groups WG)
• might be easier to project onto some sort of formal verification system

If people are interested in this approach, I can dig up the PoC branch I had and try to write up something more formal.

[JimLarson]

I worked on CEL at Google until April 2021 and I'd be happy to answer any questions about it.

[robert-zaremba]

Great use-case.

Solution: Put processed Msgs in a Transient Store

An easy solution would be to create a new Transient Store with all previous messages and responses. It would be indexed by message index, and would contain the following:

type ProcessedMsg struct {
  Msg Msg
  Resp MsgResponse // a new Any
}

Then you would need to create a new message, to include an offset prameter referencing an output from the previous message. eg:

message MsgIBCTransferWithPastPreMsgOutput {
   byte msg_offset = 1;          // 1 = previous message, 2 = prev prev message ... 
   string offset_attribute = 2; //   eg tokenOutAmount
   ... other IBC transfer attributes 
}

Accessing the store

Currently, a unique transient store is assign for every module requesting it.
However, we need a new type of transient store: one for which is created for every new transaction and common for all modules.
Two ideas how this can be accesed:

1. Each module can receive a ProcessedMsgStoreKey and access the store using the existing ctx.KVStore interface
2. Create a new method in the sdk.Context: ctx.ProcessedMsgStore() IndexedStore, where the indexed store is like KVStore but uses int32 as keys, instead of byte[].
3. We could also implement Store Accessor pattern. I was experimenting with it at: https://github.com/cosmos/cosmos-sdk/tree/robert/context-accessor

[robert-zaremba]

I've updated the design. Let me know if it makes sense and I can draft an ADR. We will need that feature at Umee as well.

[JimLarson]

Have a look at "Promise Pipelining" from the Xanadu project as prior art, especially for error handling:

https://wiki.c2.com/?PromisePipelining
http://erights.org/elib/distrib/pipeline.html