Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Enable request hedging for WebClient #806

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

Anusien
Copy link

@Anusien Anusien commented Aug 10, 2020

Hedging is a mechanism for idempotent requests to automatically start a new request after some time threshold has been reached. For example, automatically firing off a new request when the latency of an existing HTTP call is at the 95th percentile. In some kind of retry mechanism, you'd have to cancel the first request. With hedging, you instead let them all run and simply use the first one that responds. The goal is to trade off some extra network traffic for reduced tail latencies. This is something we built and have been using ourselves. We thought it would be useful to contribute back to the community.

There are two specific implementation issues that I need help from the community here to answer.

First, the implementation here uses io.micrometer:micrometer.core to track the histogram of response times (to calculate the percentile latency). spring-cloud-commons doesn't have a required dependency on that library; it's optional through org.springframework.boot:spring-boot-start-actuator. I'm at a loss whether the right thing to do is to factor out this class with the dependency, add the dependency as required, or any other compromise (use a different library; note the dependency is needed to use the class).

The second is ordering. This thing adds an ExchangeFilterFunction. There's no easy way to control where this gets added in the filter list relative to the filters added by, for example, load balancing. The solutions I came up with to control this require the end user to override both LoadBalancerWebClientBuilderBeanPostProcessor and this thing's BeanPostProcessor`.

Feature: hedge WebClient requests

Hedging is a mechanism for idempotent requests to automatically start a
new request after some time threshold has been reached. Unlike a simple
retry, hedging will use the first response returned.
@spencergibb
Copy link
Member

#481

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
2022.0.x
In progress
Development

Successfully merging this pull request may close these issues.

None yet

4 participants