High amount of requests

[ethndotsh]

I am currently hosting Quirrel using Upstash.io as my database and I am noticing that it is using ~10k requests every 20 minutes or so. Is there a way to reduce this, or a reason why? Maybe this is just while it is starting up?

[Skn0tt]

Quirrel checks Redis every couple of seconds to see if there's a new job it can execute. There is no way to reduce this right now. When I designed Quirrel (and especially http://github.com/quirrel-dev/owl, the queueing library that powers it), number of Redis requests wasn't a design criterion, as they're typically very cheap performance-wise. I'm assuming this is a problem b/c Upstash charges per command?

[ethndotsh]

Exactly. Is there a line somewhere that I can change to set it to poll, say, every couple minutes? Or would that not even work?

[Skn0tt]

Here's the line where the interval is set: https://github.com/quirrel-dev/owl/blob/f25b3952705c7e9bf0bd6b50f183b0cc366cb1c7/src/worker/job-distributor.ts#L24 There's currently no way of changing that. I'm not sure how useful it'd be to increase that -There's an upper boundary on what we can achieve by increasing this, and it might be that Upstash is just unworkable for usage with Quirrel. Could you give a rough number on how many Redis commands per hour would be acceptable to you?

[ethndotsh]

Upstash's Free tier allows for 10K commands a day. Doing a bit of simple math, you can run ~400 commands every hour, which works out to roughly one every 6 minutes. If you got out of the free tier, for $0.1 a day (100K commands) you can get it to run roughly every minute. My math might also be completely off.

[Skn0tt]

My feeling is that getting Quirrel to work with less commands per hour won't be super trivial, certainly more work than just adding a new config parameter for it. If you absolutely need this for your project, you could probably try to fiddle around with the polling mechanism yourself. If you're just looking for a cheap way to host things, then maybe DigitalOcean's Redis or a small box on Hetzner Cloud might be more cost-effective.