Should SubmissionQueue::push() also check for cq_overflow()?

[JackKelly]

My understanding - which may be wrong - is that an operation can be in one of three possible states in io_uring:

1. Waiting in the submission queue (represented as a submission queue entry)
2. In-flight (i.e. the kernel has removed the submission queue entry from the submission queue, but has not yet finished the IO).
3. Completed.

As such, when we push onto the SubmissionQueue, there's a chance that the submission queue will not be full, but we'll still overflow the completion queue (because there are still lots of operations in flight). Should the push function also check cq_overflow?

Although, if we do this, then maybe the error returned from push should discriminate between failing due to the SQ being full; and failing because the CQ is overflowing.

Or should we leave it up to the user to check cq_overflow before calling submit?

More details here.

[SUPERCILEX]

I think this is a built-in feature rather than a flag one can ask for:

IORING_FEAT_NODROP
If this flag is set, io_uring supports almost never
dropping completion events. If a completion event occurs
and the CQ ring is full, the kernel stores the event
internally until such a time that the CQ ring has room for
more entries. If this overflow condition is entered,
attempting to submit more IO will fail with the -EBUSY
error value, if it can't flush the overflown events to the
CQ ring. If this happens, the application must reap events
from the CQ ring and attempt the submit again. If the
kernel has no free memory to store the event internally it
will be visible by an increase in the overflow value on
the cqring. Available since kernel 5.5. Additionally
io_uring_enter(2) will return -EBADR the next time it
would otherwise sleep waiting for completions (since
kernel 5.19).

[JackKelly]

Thanks for your reply!

Sorry, I should've added more detail in my original post.

I think the main reason that users would need to check liburing's io_uring_cq_has_overflow() before submitting an SQE is when users configure io_uring to use a kernel thread to poll the submission queue (by setting IORING_SETUP_SQPOLL).

When using a kernel thread to poll the SQ, the user doesn't call io_uring_enter(), so the user will never see the -EBUSY error.

But I could be wrong! I haven't tried using io_uring in SQPOLL mode yet!

[SUPERCILEX]

Ah interesting. The best way to find out would be to file an issue with https://github.com/axboe/liburing and/or look at the source.