End to End Testing for LDK node implementations

[ConorOkus]

Fast, easy and reliable testing for anything that runs using LDK.

Would be nice to setup a way to automate testing different on chain scenarios with the different backends to make sure you’re fetching and feeding the data in correctly.

[TheBlueMatt]

Pretty tricky to do in a generic way - given users have their own chain sync backends and APIs (or UIs) to send and receive payments/open channels. This may end up being easier to address with a simple "at least test these scenarios" checklist. Probably (a) open channel, send/receive payment(s), force-close channel, (b) check that the force-close was claimed, etc. The really hard part is helping users ensure their reorg handling code is correct.

[jkczyz]

Is part of the motivation to ensure that a chain::Filter implementation correctly signals what data needs to be fed into chain::Listen/chain::Confirm?

[TheBlueMatt]

The motivation here is basically we should have some way of helping users test their application end-to-end, giving them some confidence that they've implemented the LDK traits and calls correctly. chain::* implementations being a key part of that.

[Kixunil]

This may be useful to you: https://github.com/Kixunil/lnpbp-testkit

[devrandom]

for completeness, here is also: https://github.com/rustyrussell/lnprototest

[Kixunil]

Hmm, it's not clear what the OP meant - testing of the LN protocol or applications using LDK. What you linked is the former, what I did is the latter.

[devrandom]

lnprototest checks that a node implementation does the right thing when you talk to it over the p2p protocol. In effect, that's a black box test, so it would likely cover a lot of the same code. I do agree that it won't necessarily cover some non p2p aspects, such as invoice handling.

[jkczyz]

Hmm, it's not clear what the OP meant - testing of the LN protocol or applications using LDK. What you linked is the former, what I did is the latter.

Background discussion: https://lightningdevkit.slack.com/archives/CSYT185MY/p1638371923206700

[devrandom]

I was thinking of making the LDK functional test suite runnable with a configurable node and invoked from outside the lightning create. It would require a custom test framework.

Cc @ksedgwic