perf: flush Record when deadline is exceeded

[lmb]

epoll: simplify timeout calculation

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

epoll: fix eventFd.add

eventFd.add hardcoded the value to add to 1 instead of using the argument.
This hasn't caused a bug since we never call add with a different value.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

perf: opportunistically check all rings for data

Waking up user space from the kernel is expensive and so the perf reader 
allows adjusting the rate at which wakeups happen. This saves CPU at the 
cost of latency: some data will remain in the buffer for longer.

The reader is an abstraction over multiple ring buffers. It only reads from
a ring buffer if it has received a wakeup from the kernel. This is wasteful
because wakeups are expensive (due to context switching and so on) but
checking a ring for contents is cheap (just an atomic load).

Change the behaviour so that we read data from any ready ring buffer 
regardless of why we were woken up.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

perf: flush Record when deadline is exceeded

Watermark and WakeupEvents allow reducing the rate at which user space is
woken up. This creates an edge case where Read() returns 
os.ErrDeadlineExceeded even though there is data in one of the rings.

This is also a problem in the ringbuf package, where we've solved this by
checking whether there is any data when encountering an ErrDeadlineExceeded.
Adopt the same approach for the perf reader.

TestPerfReaderWakeupEvents has to change since it relies on the old 
behaviour of not checking for pending data. Rework the test so that it
doesn't test the ErrDeadlineExceeded error: it's not so important that we
don't wake up early.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

[lmb]

Reworked a little bit and fixed test failures. PTAL.