Max expiration for locks

[donaldpiret]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Looking at the feature in Sidekiq enterprise (https://github.com/mperham/sidekiq/wiki/Ent-Unique-Jobs) they seem to have a maximum amount of time the lock will be valid for (eg. lock is unlocked when the job is completed or this deadline is reached).
Based on what I'm seeing in the documentation the lock_ttl option here works differently in that it describes a minimum unlock time rather than a maximum? (at least if I'm understanding the docs correctly?)

We are trying to execute jobs using sidekiq-cron regularly but ensure that no overlapping jobs are happening. I'm currently not able to figure out a configuration that will work 100% of the time to achieve this. Most of the time my jobs end up getting stuck without the ability to continue.

Describe the solution you'd like
A clear and concise description of what you want to happen.
Can we get a lock_max_ttl option that will determine what a lock's max lifetime is? eg. if this lock is still around after this amount of time assume it is stuck and clear it.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Tried until_executed, until_and_while_executing with various client and server options, but not sure I understand how these are supposed to behave for each lock type, or something really wrong is happening that is causing my jobs to always be stuck.

Additional context
Add any other context or screenshots about the feature request here.

[mhenrixon]

Versions @donaldpiret?

[donaldpiret]

sidekiq-unique-jobs (7.0.7)
sidekiq-cron (1.1.0)
sidekiq (6.2.1)
sidekiq-pro (5.2.1)

[mhenrixon]

@donaldpiret Version 7 gives you the option of a Max expiration.

You push the job, expiration is set and then the lock is removed either on success or on expiration (lock_ttl).

If you want the lock to stick around you use UntilExpired instead.

[donaldpiret]

Hi @mhenrixon if that's the way it works then perhaps the readme needs a bit of an update?

Lock TTL decides how long to wait after the job has been successfully processed before making it possible to reuse that lock.

This seems to imply that the lock is removed only after the job has completed + the lock ttl time?

[donaldpiret]

If this is indeed an issue with the docs and the behaviour is that the lock will be removed at MAX(job completion, lock_ttl), let me know and I'm happy to open a PR for the docs.

[mhenrixon]

What I said is the truth, I still have a lot of work with the docs; PRs greatly appreciated.

[pboling]

Docs around lock_ttl are still confusing. Creating a new issue.

For future googlers though, @donaldpiret's restatement above, which wasn't refuted explicitly, is incorrect, but the difference between it and @mhenrixon's explanation is subtle, so I wanted to make it as clear as possible here.

@donaldpiret's question was about a MAX:

MAX(job completion, lock_ttl)

Actual behavior according to @mhenrixon is that it is a MIN as follows:

MIN(job_completion_time, job_enqueue_time (i.e. job_push_time) + lock_ttl seconds)

Caveat with future scheduled jobs:

MIN(job_completion_time, job_scheduled_at_time + lock_ttl seconds)