Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch Estimator to use an exponential weighting #539

Merged
merged 4 commits into from Jun 3, 2023
Merged

Switch Estimator to use an exponential weighting #539

merged 4 commits into from Jun 3, 2023

Commits on Jun 3, 2023

  1. refactor estimator to use steps/sec instead of secs/step

    @afontenot
    afontenot committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    01a696c View commit details
    Browse the repository at this point in the history

  2. Refactor estimator's prev tuple into separate elements 

    It's easier to read separate state tracking fields for steps and time
than to store them together in a single tuple.

No functional change intended.
    @afontenot
    afontenot committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    7fc8620 View commit details
    Browse the repository at this point in the history

  3. Switch Estimator to use an double exponential time-based weighting 

    This is an implementation of an exponentially weighted running average
with a tuning parameter (currently a constant in this implementation):

    * the `ews` parameter is a number of seconds. Progress the Estimator
    has observed that is older than this value will receive a total
    weight of 0.1 in the rate estimation, and newer values will receive a
    total of 0.9. The default is 15 seconds.

This implementation does double smoothing by applying the running
average to itself. The result avoids undesirable instantaneous movements
in the estimate when large updates occur.

The exponential estimator works by keeping a running tally, where an
existing tally that has aged `t` seconds is reweighted such that

    weight ^ (ewa / age) = 0.1

For instance, data aged 5 seconds with a 15 second weight parameter
would receive `weight = 0.1 ^ (5/15) = 0.464`. If it then ages another
10 seconds, it would receive `weight = 0.1 ^ (10/15) = 0.215`. After
being multiplied by both weights, it would have a weight of
`0.464 * 0.215 = 0.1`, as expected.

A couple of basic features are also implemented for higher quality
estimates:

    * We divide out any weight given to data before the estimator was
    initialized. This is called "normalization" in the code, because it
    renormalizes the weights in the weighted average to sum to 1.

    * When returning an estimate, we include the time since the last
    updated was received as non-progress, which means that the estimator
    does not freeze when progress stalls.
    @afontenot
    afontenot committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    dc3aa40 View commit details
    Browse the repository at this point in the history

  4. bump version to 0.17.5

    @afontenot
    afontenot committed Jun 3, 2023
    Configuration menu
    Copy the full SHA
    1d0668d View commit details
    Browse the repository at this point in the history