New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kqueue: remove timeout from unix.Kevent() #480
Conversation
As noted in #24, reading from the kqueue won't unblock when it is closed on all platforms. This commit solves the issue by additionally watching on a pipe which is closed when Watcher is closed. The read timeout is no longer necessary.
The timeout for unix.Kevent() is causing issues; every 100ms it will do a new unix.Kevent() syscall, which isn't too efficient: even if you have just one change an hour, you will still keep calling kevent() ten times per second, resulting in a needlessly high CPU usage. Without a timeout, kevent() will block indefinitely until there are some events, which is much more efficient. We can't just remove the timout however, since we can't interrupt the kevent() call on FreeBSD and NetBSD, and it will hang forever. This issue is described in more detail here: #262 (comment) To solve this we register a new kevent() with the file descriptor set to the closepipe; when this pipe is closed an event is sent and kevent() will stop blocking. This is a rebased version of #124. Fixes #89 Fixes #237 Fixes #333 Supersedes and closes #124 Supersedes and closes #262 Supersedes and closes #334
Please add Updated test:
|
Thanks; I looked and your test and it was useful to verify this PR; I added it in a local The problem is that it takes a very long time to run on various platforms: ~25 seconds on Linux, ~40 seconds on OpenBSD (because OpenBSD is slow), etc. which is quite long given that the test of the test suite runs in ~7 seconds. Also it doesn't really test anything except by "brute force". I'm pretty sure we can write a better test for this, but need to look at this and I don't think it's needed right now. |
I'm fine with it being Darwin-specific for now. On macOS it passes in ~500ms. |
6 years in the making... I'm interested to see if this PR will finally be merged :) |
Yes, it will. The only reason it's not yet is because I wanted to give other people (e.g. fsnotify members) a chance to review it, but I'll merge it after the weekend regardless if no one does. Going forward, PRs should be reviewed and merged (or rejected if it's no good) in a reasonable timeframe (days or weeks, rather than years). |
The timeout for unix.Kevent() is causing issues; every 100ms it will do a new
unix.Kevent() syscall, which isn't too efficient: even if you have just one
change an hour, you will still keep calling kevent() ten times per second,
resulting in a needlessly high CPU usage.
Without a timeout, kevent() will block indefinitely until there are some events,
which is much more efficient. We can't just remove the timout however, since we
can't interrupt the kevent() call on FreeBSD and NetBSD, and it will hang
forever. This issue is described in more detail here:
#262 (comment)
To solve this we register a new kevent() with the file descriptor set to the
closepipe; when this pipe is closed an event is sent and kevent() will stop
blocking.
This is a rebased version of #124.
Fixes #89
Fixes #237
Fixes #333
Supersedes and closes #124
Supersedes and closes #262
Supersedes and closes #334