Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ETH): eip1559 gas fee estimator and rpcs #2051

Merged
merged 79 commits into from
May 20, 2024
Merged

Conversation

dimxy
Copy link
Collaborator

@dimxy dimxy commented Jan 17, 2024

Add eth EIP1559 priority fee estimator (WiP):

  • simple priority gas fee estimator based on eth_feeHistory call
  • loop running the estimator with blocktime period and storing values in a ctx
  • RPCs to start/stop loop for a enabled eth coin (ETH, USDT)
  • RPC to get estimated fee per gas values
  • remove refund fee from trade fee for trade_preimage rpc for swaps v1

Add tx priority fee support for withdraw and swaps, improve gas limit for swap txns:

  • add eip1559 max priorty fee and max base fee tx type 2 props to eth txns for withdraw and swaps
  • consider adding GUI notifications based on websocket streaming channel (developed in PR feat(ETH): balance event streaming for ETH #2041).
  • add alternative gas fee estimates from an API provider (a paid service) - added as test calls. In real it will be service on a mm2 eth provider
  • improve gas limit calculation to ask less gas from user: we cannot use eth_estimateGas for our etomic contract before swap started (as token spending approval is needed) so we use different consts for different swap operations
  • do real-world test and adjust new gas limit consts if needed
  • add tests for priority fee using the new dockerized geth node
  • merge eip1559 support in feat(transaction): add chain_id to Transaction rust-web3#1 and feat(tx): add EIP-1559 transaction with priority fee support mm2-parity-ethereum#1 and fix deps
  • add 'coins' param with transaction type (legacy, 1, 2) supported for different eth coins: ETH, ETC etc
  • decide where to have FeeEstimatorContext
  • decide where to store gas api provider config (currently in MM2.json)
  • make config which chains are supported for gas estimation with provider and which use only internal estimator (added 'gas_fee_estimator' in coins)
  • add eip1559 for trezor (when PR feat(hd_wallet): evm hd wallet and trezor #1979 merged)
  • improve gas fee estimates update: use eth_subscribe to get estimates when a new block comes, make a config with interval value if a gas api provides frequent updates
  • in this PR trade_preimage rpc now does not include refund fee. Make a option to return it or always return it in a dedicated field

This PR covers requirements in #1848 issue for Eth coin and ERC20 tokens, allowing more predictable priority-based gas fee and lesser gas limit for swap transactions

@dimxy dimxy added the in progress Changes will be made from the author label Jan 17, 2024
Copy link
Member

@artemii235 artemii235 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question about how the feature is supposed to work in general: is it really required to run the background loop and keep additional context? What are the benefits of this approach over using "on demand" approach?

PS, by "on demand" I mean e.g., performing external API requests and all calculations in get_eth_gas_price_estimated.

cc @shamardy

@dimxy
Copy link
Collaborator Author

dimxy commented Feb 16, 2024

I have a question about how the feature is supposed to work in general: is it really required to run the background loop and keep additional context? What are the benefits of this approach over using "on demand" approach?

This is essentially how other wallets work.
I believe calls to a gas api provider could last relatively long plus, if estimates are not expected to change until a new block comes it's no reason to ping it more often. Also the context with estimated data is shared among all eth coins/tokens. Thus gui could fetch the recent results quickly, without forwarding calls to a provider

@artemii235
Copy link
Member

I see, thanks for the explanation 🙂

Copy link
Member

@artemii235 artemii235 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next review iteration.


#[allow(dead_code)]
impl InfuraGasApiCaller {
const INFURA_GAS_API_URL: &'static str = "https://gas.api.infura.io/networks/1";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the network ID (1) should be configurable through coins file.

Copy link
Collaborator Author

@dimxy dimxy Feb 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another provider (which in fact we are going to use) supports only mainnet. I think won't support testnets for this feature. I added a check for chain_id (that it can be only mainnet) in this commit ee85d20


fn get_infura_gas_api_url() -> (String, Vec<(&'static str, &'static str)>) {
let url = format!("{}/{}", Self::INFURA_GAS_API_URL, Self::INFURA_GAS_FEES_CALL);
let headers = vec![("Authorization", INFURA_GAS_API_AUTH_TEST.as_str())];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Infura API key should be configurable, most likely through MM2.json because the key can be likely shared between multiple coins.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a config in MM2.json

let resp = slurp_url_with_headers(&url, headers).await.mm_err(|e| e.to_string())?;
if resp.0 != StatusCode::OK {
let error = format!("{} failed with status code {}", Self::INFURA_GAS_FEES_CALL, resp.0);
info!("gas api error: {}", error);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be error! instead of info!.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is propagated to the GUI. Here this logging is for debug purposes (so I have just changed it to debug) - I think we do not want to flood logs with this kind of messages

Self::BLOCKNATIVE_GAS_PRICES_CALL,
Self::BLOCKNATIVE_GAS_PRICES_PARAMS
);
let headers = vec![("Authorization", BLOCKNATIVE_GAS_API_AUTH_TEST.as_str())];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, block native AUTH token should be configurable via file, not environment variable.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a config value for the gas api in MM2.json. However I'd like to leave the api key const like it is for now. This is a temp solution, for testing gas api, until we switch to use komodo proxy instead of direct connection to the api. With the komodo proxy the authentication would be different and this const would be removed

fn calculate_with_history(fee_history: &FeeHistoryResult) -> FeePerGasEstimated {
let base_fee = *fee_history.base_fee_per_gas.first().unwrap_or(&U256::from(0));
let base_fee = u256_to_big_decimal(base_fee, ETH_GWEI_DECIMALS).unwrap_or_else(|_| BigDecimal::from(0));
let mut priority_fees = vec![];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using vec![] is overhead here, as it is always supposed to have len == 3. It can be a fixed-size array, pre-initialized with default values.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored this code

let max_priority_fee_per_gas = u256_to_big_decimal(max_priority_fee_per_gas, ETH_GWEI_DECIMALS)
.unwrap_or_else(|_| BigDecimal::from(0));
let max_fee_per_gas = base_fee.clone()
* BigDecimal::from_f64(Self::ADJUST_MAX_FEE[i]).unwrap_or_else(|| BigDecimal::from(0))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All elements of ADJUST_MAX_FEE are 1.0. Is access by index needed here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I ve seen in another wallet impl similarly used values are different. So I did this the same way to tune the values after more tests are done

let max_fee_per_gas = base_fee.clone()
* BigDecimal::from_f64(Self::ADJUST_MAX_FEE[i]).unwrap_or_else(|| BigDecimal::from(0))
+ max_priority_fee_per_gas.clone()
* BigDecimal::from_f64(Self::ADJUST_MAX_PRIORITY_FEE[i]).unwrap_or_else(|| BigDecimal::from(0)); // TODO maybe use checked ops
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All elements of ADJUST_MAX_PRIORITY_FEE are 1.0. Is access by index needed here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is to be tuned yet. Tests need to be done to adjust those coefficients for better estimations

Comment on lines 249 to 251
low: priority_fees[0].clone(),
medium: priority_fees[1].clone(),
high: priority_fees[2].clone(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe instead of vector and array, there should be even 3 separate low, medium and high variables. This will help to avoid unnecessary cloning.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored this code

Self::BLOCKNATIVE_GAS_PRICES_CALL,
resp.0
);
info!("gas api error: {}", error);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose it should be error!.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above, I suggest changing it to debug, not to flood log files. The error is sent to the gui nonetheless

Comment on lines 4653 to 4654
// To call infura use:
// let provider_estimator_fut = InfuraGasApiCaller::fetch_infura_fee_estimation();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to other comments, this should be uncommented and made configurable. I find quite confusing to have a working feature that requires code modification to be used.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a config "gas_api" in MM2 json

@laruh
Copy link
Member

laruh commented Feb 20, 2024

@dimxy please keep in mind, that PR started to have conflicts in eth.rs


/// Get latest estimated fee per gas
///
/// Param: coin ticker
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documenting function signature isn't worth, please remove them from all documentations in this PR

ref #2049 (comment)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment on lines 379 to 386
let coin = match lp_coinfind(&ctx, &req.coin).await {
Ok(Some(coin)) => coin,
Ok(None) | Err(_) => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
let coin = match coin {
MmCoinEnum::EthCoin(eth) => eth,
_ => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single match can cover both of these cases

Suggested change
let coin = match lp_coinfind(&ctx, &req.coin).await {
Ok(Some(coin)) => coin,
Ok(None) | Err(_) => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
let coin = match coin {
MmCoinEnum::EthCoin(eth) => eth,
_ => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
let coin = match lp_coinfind(&ctx, &req.coin).await {
Ok(Some(MmCoinEnum::EthCoin(eth))) => eth,
Ok(Some(_)) => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
Ok(None) | Err(_) => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};

Copy link
Collaborator Author

@dimxy dimxy Feb 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thank you for the improvement

Comment on lines 360 to 367
let coin = match lp_coinfind(&ctx, &req.coin).await {
Ok(Some(coin)) => coin,
Ok(None) | Err(_) => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
let coin = match coin {
MmCoinEnum::EthCoin(eth) => eth,
_ => return MmError::err(FeeEstimatorError::CoinNotFoundOrSupported),
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here


impl FeeEstimatorStartStopResponse {
#[allow(dead_code)]
pub fn get_result(&self) -> String { self.result.clone() }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit suggestion: I typically prefer returning the reference of the value and letting the caller decide whether a clone is needed or not. For example, if caller needs only the reference of this value, with this approach you will clone it regardless.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed to ref returned

Comment on lines 42 to 43
#[display(fmt = "Coin not activated or not a EVM coin")]
CoinNotFoundOrSupported,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we should seperate these errors to be more as they are quite different scenarios (Not activated / Not an EVM coin)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment on lines 122 to 123
/// Fee estimation update period in secs, basically equals to eth blocktime
fn get_refresh_interval() -> f64 { FEE_ESTIMATOR_REFRESH_INTERVAL as f64 }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not removing FEE_ESTIMATOR_REFRESH_INTERVAL and using const function?

like:

    const fn get_refresh_interval() -> f64 { 15. }

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

@laruh laruh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this feature!
I have some notes for the first review iteration

mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
Copy link
Member

@laruh laruh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some notes and suggestion related to module structure

mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Show resolved Hide resolved
dimxy and others added 5 commits February 24, 2024 18:29
added gas api config in mm2.conf
use platform coin for web3 conn
check chain id
* dev:
  feat(zcoin): ARRR WASM implementation (#1957)
  feat(trading-proto-upgrade): locked amounts, kmd burn and other impl (#2046)
  fix(indexeddb): set stop on success cursor condition (#2067)
  feat(config): add `max_concurrent_connections` to mm2 config (#2063)
  feat(stats_swaps): add gui/mm_version in stats db (#2061)
  fix(indexeddb): fix IDB cursor.continue_() call after drop (#2028)
  security bump for `h2` (#2062)
  fix(makerbot): allow more than one prices url in makerbot (#2027)
  fix(wasm worker env): refactor direct usage of `window` (#1953)
  feat(nft): nft abi in withdraw_nft RPC, clear_nft_db RPC (#2039)
  refactor(utxo): refactor utxo output script creation (#1960)
  feat(ETH): balance event streaming for ETH (#2041)
  chore(release): bump mm2 version to 2.1.0-beta (#2044)
  feat(trezor): add segwit support for withdraw with trezor (#1984)
Copy link
Member

@artemii235 artemii235 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't do it typically, but this time I thought that it is faster to write code rather than review comments 🙂

I have tried to simplify the estimator_loop as much as I could, please check the following diff (refactoring proposal): https://github.com/KomodoPlatform/komodo-defi-framework/compare/eip1559-gas-fee-estimate...KomodoPlatform:komodo-defi-framework:eip1559-gas-fee-estimate-refactoring-proposal?expand=1

@dimxy
Copy link
Collaborator Author

dimxy commented Feb 27, 2024

I have tried to simplify the estimator_loop as much as I could...

Thank you for this, studying...
I understand this code could be much simple. The use case I tried to implement was like that:

  • Fee estimation loop starts when user opens a page with a transaction creation and finishes when this page is closed.
  • Also, user could open concurrently 2 or more pages for different eth tokens (for tx creation), so the same estimation loop should be running and shared between those tokens. When all pages closed the loop ends.

If this is an overkill we can certainly reduce my code and just allow for GUI to handle estimation itself (when it needs it)

@artemii235
Copy link
Member

@dimxy
Yeah, I understand the idea. The proposal code actually implements it in a bit different way, considering the following:

  1. Once the loop is started for ETH, it implicitly enables estimation for all ERC20 tokens, with "platform_coin":"ETH". So there is no need to keep using_coins state in Rust. This should be documented, so GUI developers/users are aware.
  2. run_state can be simplified anyway because there seem to be only Started or Stopped state.
  3. abort_handler: AsyncMutex<Option<AbortOnDropHandle>> covers both "stop by RPC" and "application stop" cases: it is explicitly taken from FeeEstimatorContext on RPC call or dropped with the context when stop RPC is invoked. I used Arc<AsyncMutex<FeePerGasEstimated>> as fee_estimator_loop argument to ensure that it doesn't keep the additional reference to FeeEstimatorContext, preventing it from dropping on stop.
  4. Also, (just IMHO) wrapping mpsc::Receiver in Arc<Mutex> is a potential design problem. The Single Consumer pattern is not intended to be concurrently used in multiple threads (explicitly marked !Sync).

@dimxy
Copy link
Collaborator Author

dimxy commented Feb 28, 2024

4. Also, (just IMHO) wrapping mpsc::Receiver in Arc<Mutex> is a potential design problem. The Single Consumer pattern is not intended to be concurrently used in multiple threads (explicitly marked !Sync).

I understand now Arc is not good here. I guess I should have used simply Mutex. Thank you for that.

@dimxy
Copy link
Collaborator Author

dimxy commented Feb 29, 2024

@dimxy Yeah, I understand the idea. The proposal code actually implements it in a bit different way..

Yes I think it's better to use your code and let the GUI to control fee per gas estimation running

@artemii235
Copy link
Member

I understand now Arc is not good here. I guess I should have used simply Mutex. Thank you for that.

If possible, Mutex should be avoided too 🙂 The Receiver can be moved to the handling thread or future without wrappers. It has to be determined on case by case basis.

@dimxy
Copy link
Collaborator Author

dimxy commented Mar 3, 2024

I understand now Arc is not good here. I guess I should have used simply Mutex. Thank you for that.

If possible, Mutex should be avoided too 🙂 The Receiver can be moved to the handling thread or future without wrappers. It has to be determined on case by case basis.

For future reference:
I believe this property !Sync for Receiver exists for std::sync::mpsc channel kind (and I used futures::channel::mpsc for which there is no such marker).
I tried to make a test for std::sync::mpsc though: the compiler handles improper Receiver use pretty well, like wrapping it with Arc and sending Arc to a spawned thread. Besides, I guess !Sync does not prohibit sending Receiver itself to another thread (and compiler allowed to do this in my test).
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=82c1b216daddc1f104b15647739e4102)

* dev:
  fix(tests): fix failing tests (#2085)
  fix(wasm): websocket url validation (#2096)
  deps(zcoin): use librustzcash that uses the same `aes` version as mm2 (#2095)
@shamardy shamardy requested review from shamardy and laruh May 6, 2024 14:53
@shamardy
Copy link
Collaborator

shamardy commented May 6, 2024

@dimxy please fix conflicts in this PR so that we can continue with the review.

dimxy added 5 commits May 9, 2024 14:09
* dev:
  feat(app-dir): implement root application dir `.kdf` (#2102)
  fix tendermint fee calculation (#2106)
  update dockerfile (#2104)
  docs(README): remove outdated information from the README (#2097)
  fix(sia): fix sia compilation after hd wallet PR merge (#2103)
  feat(hd_wallet): utxo and evm hd wallet and trezor (#1962)
  feat(sia): initial Sia integration (#2086)
  fix(BCH): deserialize BCH header that uses KAWPOW version correctly (#2099)
  fix(eth_tests): remove ETH_DEV_NODE from tests (#2101)
  feat(coin): support nucleus as an alternative to iris HTLC (#2079)
* dev:
  fix(eth): remove my_address from sign_and_send_transaction_with_keypair (#2115)
  fix(utxo-swap): apply events occurred while taker down (#2114)
  refactor(memory): memory usage improvements (#2098)
Copy link
Member

@laruh laruh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fixes! I I have one note for now that caught my eye.

mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
dimxy added 2 commits May 13, 2024 21:13
* dev:
  feat(tendermint): pubkey-only activation and unsigned tx (#2088)
  fix(tests): set txfee for some tbtc tests (#2116)
@laruh
Copy link
Member

laruh commented May 16, 2024

@dimxy note that PR started to have conflicts 👀

@dimxy
Copy link
Collaborator Author

dimxy commented May 16, 2024

@dimxy note that PR started to have conflicts 👀

updated

shamardy
shamardy previously approved these changes May 16, 2024
Copy link
Collaborator

@shamardy shamardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing Work 🔥 ! All below comments are non-blockers!
Please add comments documenting new coin configs and new APIs, or API changes for QA team. You can add them in the tracking issue or in this PR. You can follow the way I used here to document changes #1962 (comment), #2014 (comment)

mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
mm2src/coins/rpc_command/get_estimated_fees.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/eip1559_gas_fee.rs Outdated Show resolved Hide resolved
mm2src/coins/eth/web3_transport/mod.rs Outdated Show resolved Hide resolved
mm2src/coins/lp_coins.rs Show resolved Hide resolved
mm2src/mm2_main/src/lp_swap/maker_swap.rs Show resolved Hide resolved
@@ -2477,7 +2479,7 @@ pub async fn taker_swap_trade_preimage(

let preimage_value = TradePreimageValue::Exact(my_coin_volume.to_decimal());
let my_coin_trade_fee = my_coin
.get_sender_trade_fee(preimage_value, stage)
.get_sender_trade_fee(preimage_value, stage, NO_REFUND_FEE)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

@dimxy
Copy link
Collaborator Author

dimxy commented May 17, 2024

@KomodoPlatform/qa this PR adds support for eth EIP-1559 gas priority fee, for withdrawals and swaps.

Allow eip-1559 transactions for coins
To allow eip-1559 transactions (which have type 2) there is a new "max_eth_tx_type" parameter in the coins file:

{
  "coin": "ETH",
  ...
  "chain_id": 1,
  "max_eth_tx_type": 2
}

Note, that the "chain_id" param must be also set, when the max_eth_tx_type is set to 2. (I guess we have recently set chain_id always mandatory).

Withdrawals with new priority fee per gas
For withdrawals a new WithdrawFee::EthGasEip1559 option is added with max_priority_fee_per_gas and max_fee_per_gas fields (in gwei, BigDecimal).
WithdrawFee::EthGasEip1559 variant also has a new "gas_option" enum field with EthGasLimitOption::Set(u64) or EthGasLimitOption::Calc options to set custom gas limit or make the API calculate it.
See sample.

Setting swap policy to eip-1559
GUI can set to use the eip-1559 policy for swap transactions by a new "set_swap_transaction_fee_policy" rpc, with options to use Internal, Low, Medium and High policies. 'Internal' means to create legacy transactions with gas price obtained from eth nodes (default option). Low, Medium and High are eip-1559 tx priority levels. For eip-1559 variants MM2 will be querying the configured gas provider and use the gas fee values for the policy the user set with this rpc.
There is also an "get_swap_transaction_fee_policy" rpc to query the currently set policy.

Getting suggested fee per gas
There is a gas fee estimator added to API, which returns suggested gas fees for low, medium, high priority levels, either from supported gas api providers (Infura or BlockNative) or from an internal "simple" gas fee estimator based on fee history.

To configure the gas fee provider url a new param in MM2.json was added:
"gas_api": {"provider": "Blocknative", "url": "https://api.provider.com"} or "gas_api": {"provider": "Infura", "url": "https://api.provider.com"}
There is no configuration needed for the internal "simple" gas fee estimator, it needs only web3 api configured for the platform coin.

In coins file there is a new parameter added indicating which gas api provider should be used:

{
  "coin": "ETH",
  ...
  "gas_fee_estimator": estimator_type
}

where estimator_type could be "provider" or "simple".

When gas fee estimator configured, GUI should first start it with a "start_eth_fee_estimator" rpc with a "coin" param. (There is also "stop_eth_fee_estimator" to stop it when GUI does not need it). The estimator is always started for the platform coin (ETH) although in the start param it can be a token, so the platform coin must be first enabled for estimator to work. The running estimator periodically queries the configured gas fee provider or calls the simple estimator, if the gas fee provider fails.

GUI can query the suggested gas fees with "get_eth_estimated_fee_per_gas" rpc (with "coin" param) which returns a serialised FeePerGasEstimated struct with low, medium and high levels with max fees per gas and max priority fees.

New param in sign_raw_transaction rpc to support eip-1559
The "sign_raw_transaction" rpc supports eip-1559 transaction with a new optional "pay_for_gas" field in the request params:

"tx": {
            ...
            "gas_limit": "21000",
            "pay_for_gas": {
                "tx_type": "Eip1559",
                "max_fee_per_gas": "1234.567",
                "max_priority_fee_per_gas": "1.2",
            }
        } 

See sample.
The "tx_type" param can be also set to "legacy" with old "gas_price":

"tx": {
            ...
            "gas_limit": "21000",
            "pay_for_gas": {
                "tx_type": "Legacy",
                "gas_price": "1234.567"
            }
        } 

@dimxy
Copy link
Collaborator Author

dimxy commented May 17, 2024

@KomodoPlatform/qa this PR changes trade fees as part of initiatives to solve problems with high transaction fees.
This all affects ETH only.

"trade_preimage" rpc does not include refund fee
The "trade_preimage" rpc now does not include refund fee in the transaction preimage trade fee. This is intended not to show too big transaction fee to the user, what may discourage him from starting the swap.

At the same time "max_taker_vol" and "max_maker_vol" rpc still include refund fee (what will prevent user from creation of a swap for his whole amount which he won't be able to refund back).

To discuss: maybe it is worth to add a param to the "trade_preimage" request to allow for GUI to choose to include or not the refund fee in the trade_fee. Or, maybe always return both trade and refund fees separately.

"get_trade_fee" rpc unchanged
Note: there is yet another "get_trade_fee" rpc and it was not changed (calculated based on max trade gas of 150,000). Apparently this rpc is not used in the GUI.

Gas limits adjusted for swap operations
Instead of using a constant of 150,000 multiple constants were added for various swap stages and coins/tokens.
The new constants are here.

This all is experimental at this stage and needs to be tested and discussed.

@dimxy
Copy link
Collaborator Author

dimxy commented May 19, 2024

Please add comments documenting new coin configs and new APIs, or API changes for QA team.

done in #2051 (comment)

Copy link
Member

@laruh laruh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Huge job!

@shamardy shamardy merged commit fa71adb into dev May 20, 2024
23 of 26 checks passed
@shamardy shamardy deleted the eip1559-gas-fee-estimate branch May 20, 2024 12:54
dimxy added a commit that referenced this pull request May 22, 2024
* dev:
  feat(ETH): eip1559 gas fee estimator and rpcs (#2051)
  fix(p2pk-tests): fix p2pk tests post merge (#2119)
  fix(p2pk): show and spend P2PK balance (#2053)
  fix(swap): use tmp file for swap and order files (#2118)
pub const ETH_PAYMENT: u64 = 65_000;
/// Gas limit for swap payment tx with ERC20 tokens
/// real values are 98,9K
pub const ERC20_PAYMENT: u64 = 120_000;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to set this higher, see #2135
maybe ERC20_RECEIVER_SPEND too

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this definitely should be tuned.
Should we maybe add different gas limit sets for different coins?
Or, maybe allow to override gas limit in coins file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants