Compact blinded path creation #3011

jkczyz · 2024-04-22T19:59:28Z

Add support for creating a compact BlindedPath, which consists of:

using an SCID instead of a node id in BlindedHop::encrypted_payload
using IntroductionNode::DirectedShortChannelId

The first is accomplished by specifying SCIDs when calling BlindedPath::new_for_message using a new message::ForwardNode struct. The second is through calling BlindedPath::use_compact_introduction_node. Both are called by DefaultMessageRouter.

codecov-commenter · 2024-04-22T20:07:35Z

Codecov Report

Attention: Patch coverage is 92.19858% with 11 lines in your changes are missing coverage. Please review.

Project coverage is 89.83%. Comparing base (da7a916) to head (7d85abd).

Files	Patch %	Lines
lightning/src/ln/offers_tests.rs	81.25%	0 Missing and 6 partials ⚠️
lightning/src/blinded_path/mod.rs	86.20%	3 Missing and 1 partial ⚠️
lightning/src/util/test_utils.rs	50.00%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3011      +/-   ##
==========================================
+ Coverage   89.79%   89.83%   +0.03%     
==========================================
  Files         116      116              
  Lines       96466    96560      +94     
  Branches    96466    96560      +94     
==========================================
+ Hits        86619    86741     +122     
+ Misses       7290     7255      -35     
- Partials     2557     2564       +7

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

valentinewallace

Basically LGTM

lightning/src/onion_message/messenger.rs

valentinewallace · 2024-04-23T19:13:26Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Pedantic but the convention is for setters to use the set_ prefix: https://github.com/rust-lang/rfcs/blob/master/text/0344-conventions-galore.md#gettersetter-apis and I think it would read a bit cleaner in this case.

Hmmm... this is not a setter in the normal sense, though, as we don't pass a value to set a field with. Nor is the field private, which would necessitate a setter. It also may not update the field if a value cannot be found in the network graph. Happy to go with a better name. Just not sure if a set_ prefix is appropriate here.

Still, it does seem a bit strange that we're mutating the blinded path with a fn name that sounds like it could just as well be a straight getter.

Ok, added a use_ prefix. I can see how the docs make it less clear that compact was a verb in the name.

TheBlueMatt · 2024-05-09T19:16:08Z

lightning/src/onion_message/messenger.rs

 		T: secp256k1::Signing + secp256k1::Verification
 	>(
-		&self, recipient: PublicKey, peers: Vec<PublicKey>, secp_ctx: &Secp256k1<T>,
+		&self, recipient: PublicKey, peers: Vec<PublicKey>, scid_lookup: &SL,


Rather than passing a list of our peers and a trait so that create_blinded_paths can make a callback to the caller to ask for an SCID for a given peer, shouldn't we just pass in the peers as a ForwardNode or (PublicKey, u64)?

Yeah, come to think of it that is probably cleaner. That would also allow the caller to use the compact hop representation only when desired. (e.g., in offer/refund but not in reply paths).

TheBlueMatt

Looks like this needs a rebase.

TheBlueMatt · 2024-05-13T20:14:35Z

lightning/src/ln/channelmanager.rs

+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id
+					.iter()
+					.find(|(_, channel)| channel.context().is_usable())


Do we want to sort by oldest or biggest or something?

Good idea! Oldest is probably best as this is only for onion messages, so we just want to make sure the channel announcement has been propagated in the gossip.

jkczyz · 2024-05-13T21:48:30Z

Rebased

TheBlueMatt

Basically LGTM, but IMO we really should make this optional somehow. It could happen in a followup if we want, cause we should also change how many hops we include based on if an offfer is long-lived.

TheBlueMatt · 2024-05-14T14:58:49Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Still, it does seem a bit strange that we're mutating the blinded path with a fn name that sounds like it could just as well be a straight getter.

TheBlueMatt · 2024-05-14T14:59:26Z

lightning/src/blinded_path/mod.rs

@@ -213,6 +217,30 @@ impl BlindedPath {
 			},
 		}
 	}
+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.


We should probably have some kind of discussion of how this makes paths shorter but if a channel closes will invalidate it.

TheBlueMatt · 2024-05-14T14:59:48Z

lightning/src/blinded_path/mod.rs

+				if let Some((scid, channel_info)) = node_info
+					.channels
+					.iter()
+					.find_map(|scid| network_graph.channel(*scid).map(|info| (*scid, info)))


Similar here, should we sort by size/age?

Yeah, good catch. Using the block height from the scid as it seems the timestamp in the update is really a counter specific to the channel.

TheBlueMatt · 2024-05-14T15:01:15Z

lightning/src/ln/channelmanager.rs

+			.filter(|(_, peer)| peer.latest_features.supports_onion_messages())
+			.map(|(node_id, peer)| ForwardNode {
+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id


Should we make setting this optional somehow? I feel like if I'm building a super long-term offer I may have a different preference from something being scanned right now.

Yeah... arguably we shouldn't bother with it for reply paths, either. Just not sure exactly how we want to convey it through the MessageRouter trait. Currently, the caller makes the decision for the penultimate hop using ForwardNode, but when adding more hops the MessageRouter makes the decision since it needs a NetworkGraph to find more hops. Similarly for the introduction node.

So right now it's partly an implementation concern given you need a NetworkGraph. I guess we can just add a bool parameter and it say it is best effort? Any other ideas?

I see what you mean... if there's an obvious "default behavior" that stands out, we could have a separate method, e.g. create_long_term_blinded_paths, or have a Config struct. That way we could also have a config setting for compact offers vs offers that don't need to be QR-scanned, or privacy-oriented offers that want longer blinded paths.

TheBlueMatt · 2024-05-14T15:04:21Z

lightning/src/onion_message/messenger.rs

@@ -450,7 +455,12 @@ where
 					Err(())
 				}
 			},
+		}?;
+		for path in &mut paths {
+			path.compact_introduction_node(&network_graph);


Similar here, can we make this optional on a per-offer/message basis?

jkczyz

Oops, forgot to publish these comments.

jkczyz · 2024-05-14T16:23:53Z

lightning/src/blinded_path/mod.rs

+
+	/// Attempts to a use a compact representation for the [`IntroductionNode`] by using a directed
+	/// short channel id from a channel in `network_graph` leading to the introduction node.
+	pub fn compact_introduction_node(&mut self, network_graph: &ReadOnlyNetworkGraph) {


Ok, added a use_ prefix. I can see how the docs make it less clear that compact was a verb in the name.

jkczyz · 2024-05-14T16:47:40Z

lightning/src/blinded_path/mod.rs

+				if let Some((scid, channel_info)) = node_info
+					.channels
+					.iter()
+					.find_map(|scid| network_graph.channel(*scid).map(|info| (*scid, info)))


Yeah, good catch. Using the block height from the scid as it seems the timestamp in the update is really a counter specific to the channel.

jkczyz · 2024-05-14T17:10:44Z

lightning/src/ln/channelmanager.rs

+			.filter(|(_, peer)| peer.latest_features.supports_onion_messages())
+			.map(|(node_id, peer)| ForwardNode {
+				node_id: *node_id,
+				short_channel_id: peer.channel_by_id


Yeah... arguably we shouldn't bother with it for reply paths, either. Just not sure exactly how we want to convey it through the MessageRouter trait. Currently, the caller makes the decision for the penultimate hop using ForwardNode, but when adding more hops the MessageRouter makes the decision since it needs a NetworkGraph to find more hops. Similarly for the introduction node.

So right now it's partly an implementation concern given you need a NetworkGraph. I guess we can just add a bool parameter and it say it is best effort? Any other ideas?

valentinewallace

LGTM pending maybe making the compact encoding optional. Looks like it needs another rebase as well.

lightning/src/blinded_path/mod.rs

When sending an onion message to a blinded path, the short channel id between hops isn't need in each hop's encrypted_payload since it is not a payment. However, using the short channel id instead of the node id gives a more compact representation. Update BlindedPath::new_for_message to allow for this.

Instead of passing Vec<PublicKey> to MessageRouter::crate_blinded_path, pass Vec<ForwardNode>. This way callers can include a short_channel_id for a more compact BlindedPath encoding.

Add a method to BlindedPath that given a network graph will compact the IntroductionNode as the DirectedShortChannelId variant. Call this method from DefaultMessageRouter so that Offer paths use the compact representation (along with reply paths). This leaves payment paths in Bolt12Invoice using the NodeId variant, as the compact representation isn't as useful there. WIP: Compact introduction node

jkczyz · 2024-05-14T22:49:09Z

LGTM pending maybe making the compact encoding optional. Looks like it needs another rebase as well.

Rebased now. Yeah, maybe best to do that in a follow-up as it may be a little more involved. Let me know if you have any thoughts on #3011 (comment).

jkczyz mentioned this pull request Apr 22, 2024

Compact BlindedPath representation for onion messaging #2921

Open

9 tasks

valentinewallace reviewed Apr 23, 2024

View reviewed changes

TheBlueMatt reviewed May 9, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from 084d070 to a74e84f Compare May 9, 2024 22:55

TheBlueMatt reviewed May 13, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch 2 times, most recently from 5d08b1f to f3726a7 Compare May 13, 2024 21:48

TheBlueMatt reviewed May 14, 2024

View reviewed changes

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from f3726a7 to b295e98 Compare May 14, 2024 17:17

jkczyz commented May 14, 2024

View reviewed changes

valentinewallace reviewed May 14, 2024

View reviewed changes

lightning/src/blinded_path/mod.rs Outdated Show resolved Hide resolved

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from b295e98 to 2dd24ed Compare May 14, 2024 22:42

jkczyz added 11 commits May 14, 2024 17:48

f - missed fuzz tests

af93a68

Pass ForwardNode when creating BlindedPath

409dd96

Instead of passing Vec<PublicKey> to MessageRouter::crate_blinded_path, pass Vec<ForwardNode>. This way callers can include a short_channel_id for a more compact BlindedPath encoding.

f - take the SCID of the oldest channel

193db7d

Unwrap reply_path in extract_invoice_request

2decacb

f - rename compact_introduction_node

d0510e9

f - expand docs

79331cc

f - fix docs oops

c2a37a5

f - prefer scid with smallest block height

fd14bc2

Compact introduction node test vector

7d85abd

jkczyz force-pushed the 2024-04-compact-blinded-path-creation branch from 2dd24ed to 7d85abd Compare May 14, 2024 22:48

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compact blinded path creation #3011

Compact blinded path creation #3011

jkczyz commented Apr 22, 2024 •

edited

codecov-commenter commented Apr 22, 2024 •

edited

valentinewallace left a comment

valentinewallace Apr 23, 2024

jkczyz May 9, 2024

TheBlueMatt May 14, 2024

jkczyz May 14, 2024

TheBlueMatt May 9, 2024

jkczyz May 9, 2024

TheBlueMatt left a comment

TheBlueMatt May 13, 2024

jkczyz May 13, 2024

jkczyz commented May 13, 2024

TheBlueMatt left a comment

TheBlueMatt May 14, 2024

TheBlueMatt May 14, 2024

TheBlueMatt May 14, 2024

jkczyz May 14, 2024

TheBlueMatt May 14, 2024

jkczyz May 14, 2024

valentinewallace May 14, 2024

TheBlueMatt May 14, 2024

jkczyz left a comment

jkczyz May 14, 2024

jkczyz May 14, 2024

jkczyz May 14, 2024

valentinewallace left a comment

jkczyz commented May 14, 2024

Compact blinded path creation #3011

Are you sure you want to change the base?

Compact blinded path creation #3011

Conversation

jkczyz commented Apr 22, 2024 • edited

codecov-commenter commented Apr 22, 2024 • edited

Codecov Report

valentinewallace left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz commented May 13, 2024

TheBlueMatt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jkczyz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

valentinewallace left a comment

Choose a reason for hiding this comment

jkczyz commented May 14, 2024

jkczyz commented Apr 22, 2024 •

edited

codecov-commenter commented Apr 22, 2024 •

edited