Load Shedding
#11801
Replies: 1 comment
-
Another possibility would be if it were possible to peek the number of queued requests awaiting processing. Then if that exceeds some threshold we could start dropping requests. That would be less ideal since it would require configuring that threshold, but if it's already available then it may still be workable. Thanks! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
My server occasionally gets swamped by a sudden surge in requests. Probably someone scraping, but the scrapers are getting more and more sophisticated and use a large number of IP addresses so simple per-IP throttling is largely ineffective.
Anyway, this leads to the server running behind on serving requests, and eventually it gets behind on replying to the health check pings as well. The server is then terminated for health reasons, before the auto scaling group has a chance to scale out to meet demand.
What I'm looking for is some reasonably straightforward way to shed load under these circumstances. While it would of course be possible to implement some CPU-load-based monstrosity that would gradually tune some shed-fraction as a function of excessive CPU load, that seems awfully complex and prone to weird bugs & behaviors.
The most straightforward thing I can think of is to have a timestamp applied to each incoming request when the server receives it, then when my handler gets invoked it can simply check "oh, this is a 10 second old request => 503".
But from what I can tell there's no such timestamp available in the RequestHeader (neither in its properties/methods, nor in the attribute map).
Any suggestions? Could this be added to the RequestHeader? Thanks!
Beta Was this translation helpful? Give feedback.
All reactions