Add min feerate checks #1552

dunxen · 2022-06-20T21:19:05Z

ariard

ACK ba8d54e

tnull · 2022-06-21T09:09:07Z

lightning/src/chain/package.rs

@@ -777,12 +777,15 @@ fn compute_fee_from_spent_amounts<F: Deref, L: Deref>(input_amounts: u64, predic
 	      L::Target: Logger,
 {
 	let mut updated_feerate = fee_estimator.get_est_sat_per_1000_weight(ConfirmationTarget::HighPriority) as u64;
+	assert!(updated_feerate >= FEERATE_FLOOR_SATS_PER_KW as u64);


Mh, do we really want to assert and panic here if something goes wrong?
Couldn't this just default to the floor, i.e., updated_feerate = max(updated_feerate, FEERATE_FLOOR_SATS_PER_KW)?

Just raising the question, but maybe panicking out is exactly what we want here?

Hmm yeah I get what you mean, but if we're getting strange feerates from our estimator then maybe we'd prefer to panic instead of silently defaulting? Although, I do think we need some sort of panic message at the least which is just an extra arg here. 🤷‍♂️

Would also be keen to know if this is what we normally want with these kinds of things in LDK.

I add the same thought if we should assert or panic. Not sure if we have yet consistent defensive programming recommendations.

TheBlueMatt

I'm not a huge fan of asserting on these, I agree its nice to check the users' response, but if we're gonna add code around all the fee estimation fetches, we should just do the cmp::max ourself IMO.

dunxen · 2022-06-21T16:26:47Z

if we're gonna add code around all the fee estimation fetches, we should just do the cmp::max ourself IMO.

Cool, that sounds good. Happy to do this. Do we then just get rid of the trait comments about bothering with max()?

TheBlueMatt · 2022-06-21T16:27:40Z

I think we should leave it on the trait, or at least mention that we'll always use at least 253.

ariard · 2022-06-21T17:15:20Z

I'm not a huge fan of asserting on these, I agree its nice to check the users' response, but if we're gonna add code around all the fee estimation fetches, we should just do the cmp::max ourself IMO.

Personally, I would prefer if we still wrap that check around FeeEstimator, as I presume trait implementators might be lazy, belt-and-suspenders if cheap and straightforward are nice.

TheBlueMatt · 2022-06-21T19:40:17Z

Personally, I would prefer if we still wrap that check around FeeEstimator, as I presume trait implementators might be lazy, belt-and-suspenders if cheap and straightforward are nice.

I'm not sure I undersood - are you agreeing that you'd rather see a wrapper of the FeeEstimator that does the max for us?

ariard · 2022-06-21T22:29:19Z

I'm not sure I undersood - are you agreeing that you'd rather see a wrapper of the FeeEstimator that does the max for us?

Yes, if that what you suggested as a could-be direction with your previous comment :) ?

TheBlueMatt · 2022-06-22T03:51:11Z

Heh, yea, sorry, that wasn't clear, yes, I was suggesting we just wrap the trait in some helper that does the max.

codecov-commenter · 2022-06-22T15:54:04Z

Codecov Report

Merging #1552 (fb0a015) into main (fda3819) will increase coverage by 0.19%.
The diff coverage is 100.00%.

❗ Current head fb0a015 differs from pull request most recent head 7bc6d0e. Consider uploading reports for the commit 7bc6d0e to get more accurate results

@@            Coverage Diff             @@
##             main    #1552      +/-   ##
==========================================
+ Coverage   90.81%   91.01%   +0.19%     
==========================================
  Files          80       80              
  Lines       44534    45507     +973     
  Branches    44534    45507     +973     
==========================================
+ Hits        40445    41417     +972     
- Misses       4089     4090       +1

Impacted Files	Coverage Δ
lightning/src/chain/chaininterface.rs	`96.15% <100.00%> (+96.15%)`	⬆️
lightning/src/chain/channelmonitor.rs	`92.53% <100.00%> (+1.60%)`	⬆️
lightning/src/chain/onchaintx.rs	`94.90% <100.00%> (+0.92%)`	⬆️
lightning/src/chain/package.rs	`93.04% <100.00%> (ø)`
lightning/src/ln/channel.rs	`88.75% <100.00%> (+<0.01%)`	⬆️
lightning/src/ln/channelmanager.rs	`84.88% <100.00%> (-0.21%)`	⬇️
lightning/src/ln/functional_tests.rs	`96.96% <100.00%> (+0.13%)`	⬆️
lightning/src/ln/payment_tests.rs	`98.57% <0.00%> (-0.32%)`	⬇️
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fda3819...7bc6d0e. Read the comment docs.

TheBlueMatt · 2022-06-22T17:40:33Z

lightning/src/chain/chaininterface.rs

-	///
-	/// This method can be implemented with the following unit conversions:
-	///  * max(satoshis-per-byte * 250, 253)
-	///  * max(satoshis-per-kbyte / 4, 253)


The unit conversions are still really useful.

TheBlueMatt · 2022-06-22T17:41:27Z

lightning/src/chain/chaininterface.rs

+	F::Target: FeeEstimator,
+{
+	pub(crate) fn new(fee_estimator: &'a F) -> Self
+	where


You shoul dbe able to drop the where clause here - its redundant.

TheBlueMatt · 2022-06-22T17:45:27Z

lightning/src/chain/chaininterface.rs

+
+/// Wraps a `FeeEstimator` so that any fee estimations provided by it
+/// are bounded below by `FEERATE_FLOOR_SATS_PER_KW` (253 sats/KW)
+pub(crate) struct LowerBoundedFeeEstimator<'a, F: Deref>(pub(crate) &'a F)


Instead of storing a reference to a deref to a feeestimator and constructing the lowerboundedestimator whenever we need to get fees, lets just store the full deref to a feeestimator here, and then update places in the crate to store a LowerBoundedFeeEstimator, so its always that way and its a bit harder to forget to do the wrapping.

TheBlueMatt · 2022-06-22T17:45:32Z

lightning/src/chain/chaininterface.rs

+	}
+}
+
+impl<F: Deref> FeeEstimator for LowerBoundedFeeEstimator<'_, F>


Lets not bother with the trait and just implement directly, then in the codebase make things take a LowerBoundFeeEstimator.

Yeah, will make things simpler and then there's no forgetting to create a LowerBoundFeeEstimator as you mentioned :)

ariard · 2022-06-23T18:35:59Z

lightning/src/chain/chaininterface.rs

 	fn get_est_sat_per_1000_weight(&self, confirmation_target: ConfirmationTarget) -> u32;
 }

 /// Minimum relay fee as required by bitcoin network mempool policy.
 pub const MIN_RELAY_FEE_SAT_PER_1000_WEIGHT: u64 = 4000;
+/// Minimum feerate that takes a sane approach to rounding
+pub const FEERATE_FLOOR_SATS_PER_KW: u32 = 253;


"Minimum feerate that takes a sane approach to bitcoind weight-to-vbytes rounding"

I don't know if this subtlety is documented anywhere well, so feel free to ref Rusty's commit : ElementsProject/lightning@2e687b9

TheBlueMatt · 2022-06-29T14:48:54Z

lightning/src/chain/chaininterface.rs

 	///
-	/// This method can be implemented with the following unit conversions:
+	/// This wrapped method will be implemented with the following unit conversions:


The point of listing the conversions is the user may need to convert from whatever unit their backend feerate API returns to the units here. Thus, its not that it "will" be implemented with the conversions, but that users "can" implement it using the listed conversions (but drop the max/253 bit) if they want.

Oh yeah, of course. I didn't even realise that was the main point of the comment here. I had just considered the max part. Will fix.

TheBlueMatt · 2022-06-29T14:50:58Z

lightning/src/chain/channelmonitor.rs

@@ -1129,7 +1129,7 @@ impl<Signer: Sign> ChannelMonitor<Signer> {
 		&self,
 		updates: &ChannelMonitorUpdate,
 		broadcaster: &B,
-		fee_estimator: &F,
+		fee_estimator: &LowerBoundedFeeEstimator<F>,


This method is public - let's drop the wrapper from the parameter.

Seems we run into a few tricky issues here as we cannot clone a Deref to a FeeEstimator so that we can wrap it further down the line. Unless I'm missing something subtle and rusty. The main problem is we take in &F here and in some other places and we can't move out of the shared reference when we need to wrap.

Uhhhh, right, uhhhh, uhhhh, so I think it'll work if we impl <D: Deref> FeeEstimator for D where D::Target: FeeEstimator. It doesn't currently break anything in test, dont think it'll break downstream stuff.

Alright, let me go that direction and see :)

Seems to do the trick!

In general I think we need to go that way for all of our traits, but that's a rather large change and dealing with the bindings for it sounds....un-fun.

TheBlueMatt · 2022-06-29T14:53:16Z

lightning/src/chain/chaininterface.rs

+
+/// Wraps a `Deref` to a `FeeEstimator` so that any fee estimations provided by it
+/// are bounded below by `FEERATE_FLOOR_SATS_PER_KW` (253 sats/KW)
+pub struct LowerBoundedFeeEstimator<F: Deref>(pub F)


I don't think we want this to be public at all - it should be pub(crate) instead, I think.

TheBlueMatt · 2022-06-29T14:53:29Z

lightning/src/chain/chaininterface.rs

+/// are bounded below by `FEERATE_FLOOR_SATS_PER_KW` (253 sats/KW)
+pub struct LowerBoundedFeeEstimator<F: Deref>(pub F)
+where
+	F::Target: FeeEstimator;


nit: does this really need its own line?

Think rustfmt got me here 😅. Will fix.

dunxen · 2022-07-04T19:23:08Z

Rebasing and directly modifying commits just this one last time before more review as a lot has changed from the previous revision.

TheBlueMatt · 2022-07-05T00:01:31Z

Let me know when you're ready for more review here.

TheBlueMatt · 2022-07-08T14:32:42Z

lightning/src/chain/chaininterface.rs

 	///
-	/// This method can be implemented with the following unit conversions:
+	/// The following unit conversions can be used to convert to sats/KW. Note that it is not
+	/// necessary to use max() as the minimum of 253 will be enforced by LDK:


nit: then why even mention the max?

True story. Remove this and just leave in the max in the example conversion or also get rid of that?

I'd think just skip the max but leave the conversions.

TheBlueMatt · 2022-07-08T14:37:49Z

lightning/src/chain/onchaintx.rs

@@ -667,7 +667,7 @@ impl<ChannelSigner: Sign> OnchainTxHandler<ChannelSigner> {
 			}
 		}
 		for (_, request) in bump_candidates.iter_mut() {
-			if let Some((new_timer, new_feerate, bump_tx)) = self.generate_claim_tx(height, &request, &&*fee_estimator, &&*logger) {
+			if let Some((new_timer, new_feerate, bump_tx)) = self.generate_claim_tx(height, &request, &fee_estimator, &&*logger) {


nit: you can drop the & on the fee_estimator entirely. Fewer &s good - we're already calling a fee estimator that is a reference to a reference to a reference :(

Yeah seems like quite the onion 😅

TheBlueMatt · 2022-07-08T14:56:29Z

lightning/src/ln/channelmanager.rs

@@ -3550,7 +3550,7 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
 		PersistenceNotifierGuard::optionally_notify(&self.total_consistency_lock, &self.persistence_notifier, || {
 			let mut should_persist = NotifyOption::SkipPersist;

-			let new_feerate = self.fee_estimator.get_est_sat_per_1000_weight(ConfirmationTarget::Normal);
+			let new_feerate = LowerBoundedFeeEstimator::new(&self.fee_estimator).get_est_sat_per_1000_weight(ConfirmationTarget::Normal);


Instead of peppering these everywhere, the ChannelManager::fee_estimator field should be a LowerBoundedFeeEstimator, then its very hard to screw up and forget to wrap.

Hmm, ok so no need to worry about serialisation here? Field is completely ignored in public contract?

The fee estimator itself? No, its not serialized, the user passes in a new fee estimator to the ChannelManager deserialize method.

TheBlueMatt · 2022-07-08T14:56:51Z

lightning/src/ln/channelmanager.rs

@@ -4723,7 +4723,7 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana
 					if chan_entry.get().get_counterparty_node_id() != *counterparty_node_id {
 						return Err(MsgHandleErrInternal::send_err_msg_no_close("Got a message for a channel from the wrong node!".to_owned(), msg.channel_id));
 					}
-					let (closing_signed, tx) = try_chan_entry!(self, chan_entry.get_mut().closing_signed(&self.fee_estimator, &msg), channel_state, chan_entry);
+					let (closing_signed, tx) = try_chan_entry!(self, chan_entry.get_mut().closing_signed(&&self.fee_estimator, &msg), channel_state, chan_entry);


You should be able to drop a & here?

Hmm yeah I think I can now with how FeeEstimator currently looks.

So yeah we don't need it in this case, but if we make ChannelManager::fee_estimator a LowerBoundedFeeEstimator, then it seems necessary as we're working with something that's not a Deref to a FeeEstimator.

Right, you'll need a single & here, which is a reference to a LowerBoundedFeeEstimator which holds a Deref to a FeeEstimator. Its still two references, but shouldn't need to touch this line of code, I belive?

I think the main issue here is that the signature expects &F for first argument and F: Deref where F::Target: FeeEstimator.

So it's a ref to a deref to a FeeEstimator, so that's a bit of a problem here :/
Errors with just a single & in the case where ChannelManager::fee_estimator is a LowerBoundedFeeEstimator, because of the signature.

Right, but I thought we were moving to channel.rs only ever seeing LowerBoundedFeeEstimators, never a FeeEstimator itself.

Sorry, thought that was a public API but seems like Channel itself is not. So I can go ahead and adjust the signature.

Ah! okay, wasn't sure what the confusion was. Thanks.

TheBlueMatt · 2022-07-12T15:36:01Z

lightning/src/ln/channel.rs

@@ -917,7 +917,7 @@ impl<Signer: Sign> Channel<Signer> {
 			return Err(APIError::APIMisuseError { err: format!("Holder selected channel  reserve below implemention limit dust_limit_satoshis {}", holder_selected_channel_reserve_satoshis) });
 		}

-		let feerate = fee_estimator.get_est_sat_per_1000_weight(ConfirmationTarget::Normal);
+		let feerate = LowerBoundedFeeEstimator::new(fee_estimator).get_est_sat_per_1000_weight(ConfirmationTarget::Normal);


We don't need to wrap the estimator in channel.rs anymore, no? Plus a few places further down.

TheBlueMatt

LGTM. Feel free to squash and then let's land this!

TheBlueMatt · 2022-07-12T23:33:46Z

lightning/src/ln/channel.rs

@@ -4097,7 +4099,7 @@ impl<Signer: Sign> Channel<Signer> {

 		if !self.is_outbound() {
 			if let Some(msg) = &self.pending_counterparty_closing_signed.take() {
-				return self.closing_signed(fee_estimator, &msg);
+				return self.closing_signed(&fee_estimator, &msg);


You shouldnt need to touch this line now.

`LowerBoundedFeeEstimator` is a wrapper for `Deref`s to `FeeEstimator`s that limits the get_est_sat_per_1000_weight() method to no less than 253 sats/kW.

ariard

Code Review ACK 7bc6d0e

ariard previously approved these changes Jun 21, 2022

View reviewed changes

dunxen marked this pull request as ready for review June 21, 2022 06:06

tnull reviewed Jun 21, 2022

View reviewed changes

TheBlueMatt reviewed Jun 21, 2022

View reviewed changes

dunxen dismissed ariard’s stale review via b992a6a June 22, 2022 15:29

dunxen force-pushed the 2022-06-checkminrelayfee branch 2 times, most recently from b992a6a to 5922330 Compare June 22, 2022 15:30

TheBlueMatt reviewed Jun 22, 2022

View reviewed changes

ariard reviewed Jun 23, 2022

View reviewed changes

dunxen force-pushed the 2022-06-checkminrelayfee branch 3 times, most recently from 4093743 to 1956024 Compare June 29, 2022 14:22

TheBlueMatt reviewed Jun 29, 2022

View reviewed changes

dunxen force-pushed the 2022-06-checkminrelayfee branch from 1956024 to 98ae8bf Compare July 4, 2022 19:23

dunxen force-pushed the 2022-06-checkminrelayfee branch from 7179109 to 9ec469f Compare July 8, 2022 06:00

TheBlueMatt reviewed Jul 8, 2022

View reviewed changes

TheBlueMatt reviewed Jul 12, 2022

View reviewed changes

dunxen added 2 commits July 13, 2022 15:00

Add LowerBoundedFeeEstimator to set FeeEstimator min rates

9c0c3b0

`LowerBoundedFeeEstimator` is a wrapper for `Deref`s to `FeeEstimator`s that limits the get_est_sat_per_1000_weight() method to no less than 253 sats/kW.

Make all internal signatures accept LowerBoundedFeeEstimator

7bc6d0e

dunxen force-pushed the 2022-06-checkminrelayfee branch from fb0a015 to 7bc6d0e Compare July 13, 2022 13:05

ariard approved these changes Jul 13, 2022

View reviewed changes

TheBlueMatt approved these changes Jul 13, 2022

View reviewed changes

TheBlueMatt merged commit 2a3bf03 into lightningdevkit:main Jul 13, 2022

dunxen deleted the 2022-06-checkminrelayfee branch July 18, 2022 06:54

Navigation Menu

Add min feerate checks #1552

Add min feerate checks #1552

Conversation

dunxen commented Jun 20, 2022

ariard left a comment

Choose a reason for hiding this comment

tnull Jun 21, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

dunxen commented Jun 21, 2022

TheBlueMatt commented Jun 21, 2022

ariard commented Jun 21, 2022

TheBlueMatt commented Jun 21, 2022

ariard commented Jun 21, 2022

TheBlueMatt commented Jun 22, 2022

codecov-commenter commented Jun 22, 2022 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dunxen commented Jul 4, 2022

TheBlueMatt commented Jul 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ariard left a comment

Choose a reason for hiding this comment

tnull Jun 21, 2022 •

edited

codecov-commenter commented Jun 22, 2022 •

edited