ASB failing to complete swap transactions

[guardian-beach]

Describe the bug
The asb often (perhaps even 100% of the time) is now failing to complete swap transactions when requested by a taker.

There are two common error conditions:

• JSON-RPC errors (Swap failed: JSON-RPC request failed with code -4: failed to get output distribution: JSON-RPC request failed with code -4: failed to get output distribution)
• asb simply not recognising Bitcoin lock transactions in the mempool

The result is that the swap ends up SAFELY_ABORTED instead of continuing normally. This is very annoying for the user, at best. It does not result in funds getting lost.

It is probably related to #1425 and #1432.

This happened with an older "preview" release (checked out master/HEAD). Upgraded to 0.12.3 release, will see if it persists.

Lost/trapped Funds
No

Debug logs

{"timestamp":"2023-11-02T17:26:56.944742906Z","level":"INFO","fields":{"message":"Advancing state","state":"started","rate":"0.00497738 BTC"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"},"spans":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                                                                                                                               
{"timestamp":"2023-11-02T17:26:57.394814348Z","level":"DEBUG","fields":{"message":"Found relevant Bitcoin transaction","txid":"f0f3a11bccbc90b782e077a5c52937169a5b330e2c98dc98e38941dec2b14c9c","status":"unseen"}}       
{"timestamp":"2023-11-02T17:27:57.64604526Z","level":"INFO","fields":{"message":"Advancing state","state":"bitcoin lock transaction in mempool","rate":"0.00497573 BTC"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5
","name":"swap"},"spans":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                                                                                                    
{"timestamp":"2023-11-02T17:27:57.646119074Z","level":"INFO","fields":{"message":"Waiting for Bitcoin transaction finality","txid":"f00bar","required_confirmatio
n":"1"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"},"spans":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                                          {"timestamp":"2023-11-02T17:50:00.490746763Z","level":"INFO","fields":{"message":"Waiting for Bitcoin transaction finality","txid":"f00ar","seen_confirmations":
"1","needed_confirmations":"1"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"},"spans":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                  {"timestamp":"2023-11-02T17:50:00.60624155Z","level":"INFO","fields":{"message":"Advancing state","state":"btc is locked","rate":"0.00496874 BTC"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"},"span
s":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                                                                                                                          {"timestamp":"2023-11-02T17:53:33.019406929Z","level":"ERROR","fields":{"error":"JSON-RPC request failed with code -4: failed to get output distribution"},"span":{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap
"},"spans":[{"id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5","name":"swap"}]}                                                                                                                                                  
{"timestamp":"2023-11-02T17:53:33.01958756Z","level":"ERROR","fields":{"message":"Swap failed: JSON-RPC request failed with code -4: failed to get output distribution: JSON-RPC request failed with code -4: failed to get output distribution","swap_id":"d999ebf8-618a-4c8b-9d75-bfdd9532d3b5"}}                                                                                                                                                   
{"timestamp":"2023-11-02T17:53:39.97070916Z","level":"DEBUG","fields":{"message":"A swap stopped without sending a transfer proof: request channel closed"}}  

Platform (please complete the following information):

• ASB
• Software Version: this was with an older "preview" release (checked out HEAD in the Git repo).
• Operating System: Debian