Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Track Channel Liquidity History in ProbabilisticScorer #1625

Merged
merged 4 commits into from Oct 7, 2022
Merged

Track Channel Liquidity History in ProbabilisticScorer #1625

merged 4 commits into from Oct 7, 2022

Commits on Oct 6, 2022

  1. Track history of where channel liquidities have been in the past 

    This introduces two new fields to the `ChannelLiquidity` struct -
`min_liquidity_offset_history` and `max_liquidity_offset_history`,
both an array of 8 `u16`s. Each entry represents the proportion of
time that we spent with the min or max liquidity offset in the
given 1/8th of the channel's liquidity range. ie the first bucket
in `min_liquidity_offset_history` represents the proportion of time
we've thought the channel's minimum liquidity is lower than 1/8th's
the channel's capacity.

Each bucket is stored, effectively, as a fixed-point number with
5 bits for the fractional part, which is incremented by one (ie 32)
each time we update our liquidity estimates and decide our
estimates are in that bucket. We then decay each bucket by
2047/2048.

Thus, memory of a payment sticks around for more than 8,000
data points, though the majority of that memory decays after 1,387
data points.
    @TheBlueMatt
    TheBlueMatt committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    c5eab6e View commit details
    Browse the repository at this point in the history

  2. Calculate a new penalty based on historical channel liquidity range 

    Our current `ProbabilisticScorer` attempts to build a model of the
current liquidity across the payment channel network. This works
fine to ignore channels we *just* tried to pay through, but it
fails to remember patterns over longer time horizons.

Specifically, there are *many* channels within the network that are
very often either fully saturated in one direction, or are
regularly rebalanced and rarely saturated. While our model may
discover that, when it decays its offsets or if there is a
temporary change in liquidity, it effectively forgets the "normal"
state of the channel.

This causes substantially suboptimal routing in practice, and
avoiding discarding older knowledge when new datapoints come in is
a potential solution to this.

Here, we implement one such design, using the decaying buckets
added in the previous commit to calculate a probability of payment
success based on a weighted average of recent liquidity estimates
for a channel.

For each min/max liquidity bucket pair (where the min liquidity is
less than the max liquidity), we can calculate the probability that
a payment succeeds using our traditional `amount / capacity`
formula. From there, we weigh the probability by the number of
points in each bucket pair, calculating a total probability for the
payment, and assigning a penalty using the same log-probability
calculation used for the non-historical penalties.
    @TheBlueMatt
    TheBlueMatt committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    6852ea9 View commit details
    Browse the repository at this point in the history

  3. Track a reference to scoring parameters in DirectedChannelLiquidity 

    This simplifies adding additional half lives in
DirectedChannelLiquidity by simply storing a reference to the full
parameter set rather than only the single current half-life.
    @TheBlueMatt
    TheBlueMatt committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    ec68f13 View commit details
    Browse the repository at this point in the history

  4. Decay historical liquidity tracking when no new data is added 

    To avoid scoring based on incredibly old historical liquidity data,
we add a new half-life here which is used to (very slowly) decay
historical liquidity tracking buckets.
    @TheBlueMatt
    TheBlueMatt committed Oct 6, 2022
    Configuration menu
    Copy the full SHA
    c8fb859 View commit details
    Browse the repository at this point in the history