Attributable errors (feature 36/37)

[joostjager]

Error attribution is important to properly penalize nodes after a payment failure occurs. The goal of the penalty is to give the next attempt a better chance at succeeding. In the happy failure flow, the sender is able to determine the origin of the failure and penalizes a single node or pair of nodes.

Unfortunately it is possible for nodes on the route to hide themselves. If they return random data as the failure message, the sender won't know where the failure happened.

This PR proposes a new failure message format that lets each node commit to the failure message. If one of the nodes corrupts the failure message, the sender will be able to identify that node.

For more information, see https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-October/003723.html.

LND implementation: lightningnetwork/lnd#7139

Note: the PR is currently authored as a change rather than an addition. Later on it can be converted either to an extension bolt or merged into the main spec with if..then..else sentences.

[thomash-acinq]

I've started implementing it in eclair, do you have some test vectors so we can check that we are compatible?
The design seems good to me, but as I've said previously, I think keeping hop payloads and hmacs for 8 nodes only (instead of 27) is enough for almost all use cases and would give us huge size savings.

[joostjager]

I don't have test vectors yet, but I can produce them. Will add them to this PR when ready.

Capping the max hops at a lower number is fine to me, but do you have a scenario in mind where this would really make the difference? Or is it to more generally that everything above 8 is wasteful?

[joostjager]

@thomash-acinq added a happy fat error test vector.

[joostjager]

I think this big gap in the bits has emerged here because of tentative spec changes that may or may not make it. Not sure why that is necessary. I thought for unofficial extensions, the custom range is supposed to be used?

I can see that with unofficial features deployed in the wild, it is easier to keep the same bit when something becomes official. But not sure if that is worth creating the gap here? An alternative is to deploy unofficial features in the custom range first, and then later recognize both the official and unofficial bit. Slightly more code, but this feature list remains clean.

[joostjager]

Added fat error signaling to the PR.

[thomash-acinq]

I've spent a lot of time trying to make the test vector pass and I've finally found what was wrong:
In the spec you write that the hmac covers

• failure_len, failuremsg, pad_len and pad.

• The first y+1 payloads in payloads. For example, hmac_0_2 would cover all three payloads.

• y downstream hmacs that correspond to downstream node positions relative to x. For example, hmac_0_2 would cover hmac_1_1 and hmac_2_0.

implying that we need to concatenate them in that order. But in your code you follow a different order:

// Include payloads including our own.
_, _ = hash.Write(payloads[:(NumMaxHops-position)*payloadLen])

// Include downstream hmacs.
var hmacsIdx = position + NumMaxHops
for j := 0; j < NumMaxHops-position-1; j++ {
	_, _ = hash.Write(
		hmacs[hmacsIdx*sha256.Size : (hmacsIdx+1)*sha256.Size],
	)

	hmacsIdx += NumMaxHops - j - 1
}

// Include message.
_, _ = hash.Write(message)

I think the order message + hop payloads + hmacs is more intuitive as it matches the order of the fields in the packet.

[joostjager]

Oh great catch! Will produce a new vector.

[joostjager]

@thomash-acinq updated vector

[joostjager]

Updated LND implementation with sender-picked fat error structure parameters: lightningnetwork/lnd#7139

[thomash-acinq]

Eclair has a basic implementation of the latest version ready: ACINQ/eclair#2519
It doesn't signal the feature yet (conflict with dual funding) and does not make use of attributable errors but should relay them correctly (compatible with the test vector).

[thomash-acinq]

We need to add requirements for the reader.
It may also be useful to create a new failure code for when the downstream node sent us a legacy error.

[joostjager]

Requirement added for the reader.

[joostjager]

It may also be useful to create a new failure code for when the downstream node sent us a legacy error.

I don't think we can know this, can we?

[thomash-acinq]

We can't always know for sure but the size of the packet we receive should be a good indicator, legacy error being much shorter than attributable errors.
What do you do if the downstream node returns a legacy error (or any error that's too short)? There are no payloads and hmacs to shift.

[joostjager]

Ah I see. In lightningnetwork/lnd#7139, a properly-structured all-zeroes (attributable) failure message is returned in this case. With that, the sender at least has an indication of the offending node's position, but the message itself is not defined.

Good suggestion to make that a specific error.

[thomash-acinq]

The spec should say what to do if

• the onion is invalid,
• the error to relay is too short.

I've updated my implementation to match yours:

• default to legacy errors,
• pretend we've received all zeros.
  This seems like an acceptable solution as it preserves the attributability of the error.

We could also consider an alternative system where the use-attributable-errors bit is in the update_add_htlc message instead of being inside the onion. That way we always have access to it even if the onion is invalid (or if we decide to reject the HTLC before even reading the onion, for instance if the CLTV is too low).
And by adding a new error message to wrap a legacy error, we could even get partial attributability if the first nodes of the route support attributable errors but not the last ones.

[joostjager]

We could also consider an alternative system where the use-attributable-errors bit is in the update_add_htlc message instead of being inside the onion. That way we always have access to it even if the onion is invalid (or if we decide to reject the HTLC before even reading the onion, for instance if the CLTV is too low).

The idea would be to let all nodes just carry forward the use-attributable-errors bit to the node downstream? It would save some bytes in the onion too, although at some point in the future those bytes would probably be freed up anyway when it is decided that legacy failures are no longer supported.

I do wonder if this opens up attack vectors. A node forwarding an htlc and flipping use-attributable-errors from false to true, or vice versa.

And by adding a new error message to wrap a legacy error, we could even get partial attributability if the first nodes of the route support attributable errors but not the last ones.

The legacy error is wrapped and the sender can interpret it, even though it went through a few attributable error nodes? I've also been thinking about this, but I doubted that it is worth the extra complexity. Given enough time, most nodes will be upgraded and mixed paths aren't needed any longer. Attributable errors are not a critical hot fix, so I think we've the time for that?

[joostjager]

I came across the algorithm for hmac based one time passwords: https://en.wikipedia.org/wiki/HMAC-based_one-time_password#Definition

Interestingly they use a more complicated way to truncate the hash (truncate(MAC)). It isn't clear to me that this makes it any more secure, but perhaps it is defense in depth?

https://datatracker.ietf.org/doc/html/rfc4226#section-5.3