Allow Extending Instance and Code TTL With Separate Values on the Host Environment For More Cost-Efficient Implementations

[heytdep]

Starting here the discussion for stellar/rs-soroban-env#1355.

Original doc: https://docs.google.com/document/d/1f2hWD4LRL9A27rGImvavbBCiPUa3mYIXpcxtTfxFsLI/edit?usp=sharing

This proposal aims to introduce a change within the soroban host environment and SDK to enable the guest environment to bump the instance and the contract code with different ttl and threshold values. The goal of this proposal is to enable for better distribution of rent costs among binaries designed to be associated with more than one instance.

Motivation

Currently, when bumping/extending ttl of the instance, the guest environment can call a host function that will bump by the same amount both the contract instance entry and the linked contract code entry. In decentralized contracts, the contract itself bumps its lifetime from within the code with certain thresholds with the idea of costs being distributed among users.

Extending the life for contract code entries is between the most expensive costs of an invocation due to the large size the binary occupies on the ledger, and there are numerous situations where a contract code entry is referenced by more than one instance, thus not allowing to bump separately the contract code and the instance prevents implementing a more efficient lifetime bump logic.

For example, think of a liquidity pool contract, used by hundreds (or thousands) of actively bumped contract instances. The instance of a single pool contract is bumped by all the users of that contract, but the code entry is bumped by all the users of all the pool contracts, so extending the lifetime of the code separately (by less) would make up for a better distribution of the fees across the network.

Impacted Components

Only WASM contract executables are impacted by this as SAC’s don’t have any binary associated on the ledger and their logic is hardcoded in the environment.

Implementation

The implementation is fairly simple, and the current implementation is based on some assumptions, mainly that:

• the user will still want to update the code and instance together, just with the possibility of specifying different thresholds and time to live.
• if bumping the code separately we also always want to accept a separate threshold (i.e if bumping instance and code separately, besides providing two ttls you are also providing two thresholds).

Thus the extend_contract_instance_and_code_ttl_from_contract_id function (not part of the VmCaller trait) would be modified to accept a threshold and ttl for the code entry as optional u32s. If the options wrap u32s then the threshold/ttl for the code are those specified, else they default to the ones specified for the instance:

...
ContractExecutable::Wasm(wasm_hash) => {
                let extend_to = extend_code_to.unwrap_or(extend_instance_to);
                let threshold = code_threshold.unwrap_or(instance_threshold);
...

The env.json has been modified as follows, but it could also be wise to keep the current exports as is and add new exports for these two host functions:

{
                    "export": "a",
                    "name": "extend_current_contract_instance_and_code_ttl_separately",
                    "args": [
                        {
                            "name": "instance_threshold",
                            "type": "U32Val"
                        },
                        {
                            "name": "code_threshold",
                            "type": "U32Val"
                        },
                        {
                            "name": "extend_instance_to",
                            "type": "U32Val"
                        },
                        {
                            "name": "extend_code_to",
                            "type": "U32Val"
                        }
                    ],
                    "return": "Void",
                    "docs": "Same as extend_current_contract_instance_and_code_ttl but with the ability to extend code and instance with different lifetimes and thresholds."
                },
                {
                    "export": "b",
                    "name": "extend_contract_instance_and_code_ttl_separately",
                    "args": [
                        {
                            "name": "contract",
                            "type": "AddressObject"
                        },
                        {
                            "name": "instance_threshold",
                            "type": "U32Val"
                        },
                        {
                            "name": "code_threshold",
                            "type": "U32Val"
                        },
                        {
                            "name": "extend_instance_to",
                            "type": "U32Val"
                        },
                        {
                            "name": "extend_code_to",
                            "type": "U32Val"
                        }
                    ],
                    "return": "Void",
                    "docs": "Same as extend_contract_instance_and_code_ttl but with the ability to extend code and instance with different lifetimes and thresholds."
                },

[leighmcculloch]

Discussing the motivation:

Extending the life for contract code entries is between the most expensive costs of an invocation due to the large size the binary occupies on the ledger

This part of the problem seems like it would discourage a contract from bumping itself at all, and create behaviours around bumping that aren't attached to just the needs of the contract user.

The problem is worse the larger the contract is.

It seems very reasonable to me to decouple the contract instance and contract code bump. There might be reasons to couple them that are driven by the implementation, but from a conceptual and logical perspective it seems unnecessary.

The contract code is a dependency of a contract instance, and the env doesn't require bumps of the instance to bump other dependencies like persistent storage, or other contracts that it sub-invokes.

extending the lifetime of the code separately (by less) would make up for a better distribution of the fees across the network.

I don't think separating the two will create instance fairness for who pays for the wasm bump. Presumedly a user of a contract instance that is more frequently used is likely to need to pay the wasm bump anyway. But by coupling the instance and code together the env doesn't give developers the opportunity to do something else.

[leighmcculloch]

Replying to #1447 (comment) here:

would discourage a contract from bumping itself at all, and create behaviours around bumping that aren't attached to just the needs of the contract user.

Not sure I follow this, what do you mean by discouraging the contract not to bump itself? (the code or the instance?)

@heytdep If a contract developer really doesn't want to bump the wasm, I think their only option is to not bump the contract at all within the contract and hope users figure it out.

[heytdep]

Ah yeah absolutely bumping the instance and never the code doesn't really make sense, thanks for clarifying.

[leighmcculloch]

Discussing the proposed implementation:

If the instance and code TTLs are to be decoupled, I think we should decouple them more completely, so that there are two host functions for bumping each one individually. The interface wouldn't require understanding what the optional values are. And I don't see any need to always potentially bump the code which the proposal does.

Today the interface includes:

• extend_current_contract_instance_and_code_ttl(threshold: U32Val, extend_to: U32Val)
• extend_contract_instance_and_code_ttl(contract: AddressObject, threshold: U32Val, extend_to: U32Val)

We could extend it by adding functions for each individually so that they are truly decoupled:

• extend_current_contract_instance_ttl(threshold: U32Val, extend_to: U32Val)
• extend_current_contract_code_ttl(threshold: U32Val, extend_to: U32Val)
• extend_contract_instance_ttl(contract: AddressObject, threshold: U32Val, extend_to: U32Val)
• extend_contract_code_ttl(contract: AddressObject, threshold: U32Val, extend_to: U32Val)

[heytdep]

Do you think that these functions should have a new export or be placed "next to" the related functions by renaming other existing exports (as I did in the pr).

Also, if we extend the env with those functions, wouldn't it make sense to remove the extend_contract_instance_and_code_ttl and current contract functions from the env alltogether and move the logic (of coupling the bumps together) to the SDK?

[heytdep]

Hi @leighmcculloch thanks for the reply.

would discourage a contract from bumping itself at all, and create behaviours around bumping that aren't attached to just the needs of the contract user.

Not sure I follow this, what do you mean by discouraging the contract not to bump itself? (the code or the instance?)

I don't think separating the two will create instance fairness for who pays for the wasm bump. Presumedly a user of a contract instance that is more frequently used is likely to need to pay the wasm bump anyway. But by coupling the instance and code together the env doesn't give developers the opportunity to do something else.

Yes the user of an instance that is more frequently used will for sure also need to pay for the wasm bump anyways, and that's one of the main reasons why the proposed implementation still keeps the code and instance bump within the same env function (but I'm not opposed to having these decoupled and having the SDK doing the work of coupling them which might be a cleaner VM design). However, I do believe that allowing to specify separate thresholds and lifetimes would make up for better distribution of the costs: as of now, by bumping the instance + code with the same thresholds + ttl, we're essentially considering state shared by a contract (the instance storage map) + state shared by (potentially) many other contracts (the associated contract code) as the same thing when they aren't, mainly because the state shared by more contracts is obviously attempted to be bumped more (so it'd make more sense to lower the ttl and the threshold for it).

If the developer can specify diverse ttl and threshold amounts for the code and instance when bumping, they can actually better distribute the costs for the users, especially if these parameters are dynamic (i.e can be changed by an admin in state and adapt to the number of active instances).

Wdyt?

[leighmcculloch]

A more diverse but still partially coupled TTL and threshold is just one approach developers may take. By coupling them in anyway I think the environment is being a little too restrictive and unnecessarily prescriptive. If separate host functions are added, developers can still bump both as you say, but the freedom to bump only one will also exist.

[heytdep]

Yeah I agree, I started off the impl in order not to change the existing structure too much but it's cleaner to just separate this. Leaving a comment now on your proposed interface extension

[dmkozh]

From the env maintenance standpoint this is a fairly trivial change, so I see no harm adding it - the original design is indeed too restrictive.

Given that there is no need to make any XDR changes/link any new libraries to Core/meter any new code paths, I believe the cost of this change is trivial enough to just include it into the next protocol without dwelling too much on the exact impact - the change appears to be useful and cheap enough, so there is really no real tradeoff between the implementation/maintenance code and usability.

I would probably only consider reducing the number of functions to just 2 and remove the variants with the current contract. The added cost of calling this with the current contract seems marginal enough to me.

[leighmcculloch]

remove the variants with the current contract

Would the idea be to pass ScVal::Void or similar as the contract ID to trigger this for the current contract? Or would the developer need to retrieve the current contract Address and then use that?

[heytdep]

Yes I believe that it would be the SDK retrieving the contract address and pass it directly.

[heytdep]

@dmkozh @leighmcculloch pushed the new changes to the PR, mainly:

• updated the env interface to support extend_contract_instance_ttl and extend_contract_code_ttl + gated the "current contract" functions to max_supported_protocol set to 21, but it may be worth just setting 20 for these.
• added testutils to the host and storage to retrieve entries with lifetimes to test these
• added lifetime_extension test for instance and code

[YOGINIMAHIMA1]

can anybody clear what's the use of this function here-extend_current_contract_instance_and_code_ttl(threshold: U32Val, extend_to: U32Val)

[namankumar]

Based on the community support received by this topic, as evidenced by the number of upvotes received by this discussion as well as the engagement in the protocol meeting, a CAP has been written for it.

@heytdep do you want to review and give your approval (or suggest changes!): https://github.com/stellar/stellar-protocol/blob/master/core/cap-0053.md

[otherdimension]

Hi when is next zoom meet or other app audio video talk?

…

On Mon, 11 Mar 2024 at 23:31, Naman Kumar ***@***.***> wrote: Based on the community support received by this topic, as evidenced by the number of upvotes received by this discussion as well as the engagement in the protocol meeting, a CAP has been written for it. @heytdep <https://github.com/heytdep> do you want to review and give your approval (or suggest changes!): https://github.com/stellar/stellar-protocol/blob/master/core/cap-0053.md — Reply to this email directly, view it on GitHub <#1447 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AOTEJDSBQNT7BAPMLAMAE2TYXZEEXAVCNFSM6AAAAABDZBD4KSVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DONJSG4YDQ> . You are receiving this because you are subscribed to this thread.Message ID: <stellar/stellar-protocol/repo-discussions/1447/comments/8752708@ github.com>