allow functional tests to be used externally with a dynamic signer factory #3016

devrandom · 2024-04-24T12:09:20Z

dynamic signer factory
DynSigner
apply xtest macros
PoC external usage

mut-global/src/lib.rs

codecov-commenter · 2024-04-24T13:31:37Z

Codecov Report

Attention: Patch coverage is 90.32258% with 48 lines in your changes are missing coverage. Please review.

Project coverage is 89.83%. Comparing base (0e22b12) to head (445e611).
Report is 24 commits behind head on main.

Files	Patch %	Lines
lightning/src/util/dyn_signer.rs	88.82%	18 Missing and 1 partial ⚠️
lightning/src/util/test_channel_signer.rs	7.14%	13 Missing ⚠️
ext-test-macro/src/lib.rs	68.00%	8 Missing ⚠️
lightning/src/util/test_utils.rs	84.21%	3 Missing ⚠️
lightning/src/events/mod.rs	50.00%	2 Missing ⚠️
lightning/src/ln/onion_utils.rs	66.66%	2 Missing ⚠️
lightning/src/util/mut_global.rs	96.66%	1 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3016      +/-   ##
==========================================
+ Coverage   89.15%   89.83%   +0.68%     
==========================================
  Files         118      121       +3     
  Lines       97678   102001    +4323     
  Branches    97678   102001    +4323     
==========================================
+ Hits        87080    91637    +4557     
+ Misses       8357     8166     -191     
+ Partials     2241     2198      -43

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

tnull · 2024-04-24T14:40:10Z

ext-test-macro/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "ext-test-macro"


I wonder if we should call this more generally ldk-macros? Currently, we use bdk-macros to provide a dual blocking/async interface in lightning-transaction-sync. However, the bdk-macros crate is basically going away with BDK 1.0, so we might want to include the corresponding functionality here?

(cc @TheBlueMatt)

devrandom · 2024-04-24T15:45:07Z

lightning/src/sign/mod.rs

@@ -727,6 +725,32 @@ impl HTLCDescriptor {
 /// A trait to handle Lightning channel key material without concretizing the channel type or
 /// the signature mechanism.
 pub trait ChannelSigner {
+	/// Returns the commitment seed for the channel.


maybe create a separate trait for these

Yea, sgtm, some kind of extension trait should do.

devrandom · 2024-04-24T15:46:01Z

lightning/src/sign/mod.rs

+		self.commitment_seed
+	}
+
+	fn counterparty_pubkeys(&self) -> Option<&ChannelPublicKeys> {


move to trait

lightning/src/util/mut_global.rs

devrandom · 2024-04-25T11:36:57Z

lightning/src/util/dyn_signer.rs

+        htlc: &HTLCOutputInCommitment,
+        secp_ctx: &Secp256k1<secp256k1::All>,
+    ) -> Result<Signature, ()>  {
+        EcdsaChannelSigner::sign_justice_revoked_htlc(&*self.inner, justice_tx, input, amount, per_commitment_key, htlc, secp_ctx)


name clashes are inconvenient, result in this need to disambiguate, and can't use delegate! macro

TheBlueMatt · 2024-05-08T14:50:35Z

lightning-invoice/src/utils.rs

-		let cross_node_seed = [44u8; 32];
-		chanmon_cfgs[1].keys_manager.backing = PhantomKeysManager::new(&seed_1, 43, 44, &cross_node_seed);
-		chanmon_cfgs[2].keys_manager.backing = PhantomKeysManager::new(&seed_2, 43, 44, &cross_node_seed);
+		chanmon_cfgs[1].keys_manager.backing = make_dyn_keys_interface(&seed_1);


I'm surprised these tests pass? The point is to have the same cross_node_seed across the nodes.

TheBlueMatt · 2024-05-08T14:54:07Z

lightning/src/ln/channelmanager.rs

@@ -1981,9 +1978,6 @@ where
 	/// required to access the channel with the `counterparty_node_id`.
 	///
 	/// See `ChannelManager` struct-level documentation for lock order requirements.
-	#[cfg(not(test))]


In general I'd prefer to keep these kinds of things if possible. They're kinda verbose and dumb but I don't really want internal things leaking and someone starting to rely on them being usable across the crate.

TheBlueMatt · 2024-05-08T14:57:24Z

lightning/src/sign/mod.rs

@@ -727,6 +725,32 @@ impl HTLCDescriptor {
 /// A trait to handle Lightning channel key material without concretizing the channel type or
 /// the signature mechanism.
 pub trait ChannelSigner {
+	/// Returns the commitment seed for the channel.


Yea, sgtm, some kind of extension trait should do.

TheBlueMatt · 2024-05-08T15:00:11Z

lightning/src/util/mod.rs

+pub mod dyn_signer;
+
+#[cfg(feature = "std")]
+pub mod mut_global;


Do these need to be visible outside of test builds?

TheBlueMatt · 2024-05-08T15:02:02Z

lightning/Cargo.toml

+delegate = "0.12.0"
+ext-test-macro = { path = "../ext-test-macro" }


Similar here, can we make these only a part of _test_utils builds only?

TheBlueMatt · 2024-05-08T15:02:55Z

lightning/src/util/dyn_signer.rs

+}
+
+impl NodeSigner for DynKeysInterface {
+	delegate! {


Honestly not convinced this actually saves enough typing to be worth it, but whatever.

TheBlueMatt · 2024-05-08T15:18:50Z

lightning/src/util/dyn_signer.rs

+
+#[cfg(not(taproot))]
+/// A super-trait for all the traits that a dyn signer backing implements
+pub trait DynSignerTrait: EcdsaChannelSigner + Send + Sync {}


Is this all because we need Writeable for the channelsigner? I think we can remove the Writeable bound entirely now! We started ignoring the bytes read in OnchainTx in 0.0.113, and according to our release notes LDK objects written by 0.0.119 are no longer expected to be readable in 113 anyway, so we can just wholesale remove the Writeable bound and move on.

devrandom commented Apr 24, 2024

View reviewed changes

mut-global/src/lib.rs Outdated Show resolved Hide resolved

devrandom force-pushed the 2024-04-ext-test branch from 1a142e1 to 4077bc2 Compare April 24, 2024 14:39

tnull reviewed Apr 24, 2024

View reviewed changes

devrandom force-pushed the 2024-04-ext-test branch from 4077bc2 to 3a1986c Compare April 24, 2024 15:32

devrandom commented Apr 24, 2024

View reviewed changes

lightning/src/util/mut_global.rs Outdated Show resolved Hide resolved

devrandom force-pushed the 2024-04-ext-test branch 2 times, most recently from 9d4e233 to f341a0c Compare April 25, 2024 09:02

devrandom commented Apr 25, 2024

View reviewed changes

devrandom force-pushed the 2024-04-ext-test branch 6 times, most recently from 87aadef to 4efd374 Compare April 26, 2024 08:19

devrandom marked this pull request as ready for review April 26, 2024 08:20

devrandom force-pushed the 2024-04-ext-test branch from 4efd374 to f3256c1 Compare April 26, 2024 09:21

devrandom added 3 commits April 26, 2024 11:34

fix non-local macro in doctest

ac2eae7

fix check-cfg-flags.py

2c7c4b8

externally visible test macro

8764062

devrandom force-pushed the 2024-04-ext-test branch from f3256c1 to 3c454d0 Compare April 26, 2024 09:35

devrandom added 6 commits April 27, 2024 13:31

mut-global utility

e19a406

introduce DynSigner and TestSignerFactory

37c040d

taproot

6e074ee

expose one functional test

8eb8b4d

demo two exposed functional tests

3805ac1

expose rest of functional tests

ef8f13d

devrandom force-pushed the 2024-04-ext-test branch from 3c454d0 to ef8f13d Compare April 27, 2024 11:39

devrandom added 3 commits April 27, 2024 14:21

f mut_global

affc49e

f fuzz DynSigner

c6114cc

f invoice

445e611

TheBlueMatt reviewed May 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

allow functional tests to be used externally with a dynamic signer factory #3016

allow functional tests to be used externally with a dynamic signer factory #3016

devrandom commented Apr 24, 2024 •

edited

codecov-commenter commented Apr 24, 2024 •

edited

tnull Apr 24, 2024 •

edited

devrandom Apr 24, 2024

TheBlueMatt May 8, 2024

devrandom Apr 24, 2024

devrandom Apr 25, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 8, 2024

TheBlueMatt May 10, 2024

		delegate = "0.12.0"
		ext-test-macro = { path = "../ext-test-macro" }

allow functional tests to be used externally with a dynamic signer factory #3016

Are you sure you want to change the base?

allow functional tests to be used externally with a dynamic signer factory #3016

Conversation

devrandom commented Apr 24, 2024 • edited

codecov-commenter commented Apr 24, 2024 • edited

Codecov Report

tnull Apr 24, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

devrandom commented Apr 24, 2024 •

edited

codecov-commenter commented Apr 24, 2024 •

edited

tnull Apr 24, 2024 •

edited